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In this issue 


In the first paper of this issue of Survey Methodology, Estevao and Sandal consider the problem of 
calibration estimation in the context of two-phase sampling. The contributions of the paper include the 
choice of initial weights in the calibration procedure as well as the important problem of variance 
estimation. New variance estimators are proposed and results from a simulation study show that the 
proposed variance estimators are more efficient than the traditional ones. 

Next, Li and Valliant investigate the problem of the detection of influential units in linear regression 
analysis of survey data. They first give an expression for the hat matrix and its associated leverages 
(diagonal of the hat matrix) when a weighted least squares technique is used to estimate model parameters. 
They then propose a decomposition of the leverages and highlight that the leverage for a given unit can be 
large when either its survey weight is large or its vector of explanatory variables is far from the center. 
They illustrate the effect of influential units on both ordinary and weighted least squares using a numerical 
example. 

Beaumont and Bocci propose a bootstrap methodology for testing hypotheses about a vector of 
unknown model parameters when the sample has been drawn from a finite population. The technique uses 
model-based test statistics that incorporate the survey weights and can usually be obtained easily using 
standard software packages. Using a simulation study the authors show that the proposed method performs 
similarly to the Rao-Scott procedure, and better than the Wald and Bonferroni procedures when testing 
hypotheses about a vector of linear regression model parameters. 

The paper by Park, Choi and Choi present an interesting approach to nonresponse. Studies have shown 
that the voting behaviour of the undecided voters can have a significant impact on the final result of an 
election and that by considering these undecided voters, the accuracy of election forecasting can be 
improved. The authors present two Bayesian models whose priors depend on information from both 
respondents and undecided. They analyze an incomplete two-way contingency table using four sets of data 
from the 1998 Ohio state polls to illustrate how to use and interpret estimation results for the elections. 

Ghosh, Kim, Sinha, Maiti, Katzoff and Parsons develop hierarchical and empirical Bayes methods for 
estimation of proportions in small domains using unit-level models. They propose a hierarchical Bayes 
analogue of the generalized linear mixed model to obtain posterior means and posterior standard errors of 
the population small domain proportions. Using an approach based on the theory of optimal estimating 
functions, they also obtain emprical Bayes estimators and corresponding asymptotic mean square error 
estimators. The methods are illustrated using data from the National Health Interview Survey (NHIS) to 
obtain small domain estimates of the proportions of Asians without health insurance. 

In the McElroy and Holan paper, the problem of testing for residual seasonality in seasonally adjusted 
data is investigated. The authors propose a statistical significance test for peaks in the spectral density of the 
time series under consideration that is indicative of seasonality. The theory of the proposed method 
developed and is illustrated and compared with existing methods through both simulation and empirical 
studies. 

Gabler and Lahiri provide a model-assisted justification of the traditional interviewer variance formula 
for equal probability sampling with no spatial clustering. They then obtain, in the context of a complex 
sampling design, a definition of interviewer variability that appropriately accounts for unequal probabilities 
of selection and spatial clustering. They also propose a decomposition of total effects into effects due to 
weighting, spatial clustering and interviewers. Their results can help to more effectively understand and 
control sources of variability. 


2 In this issue 


In their paper, Schouten, Cobben and Bethlehem investigate the problem of assessing the similarity 
between the response to a survey and the sample or population under investigation. They propose a 
representativeness indicator to replace response rates as a quality indicator for the impact of nonresponse 
bias. This indicator, called the R-indicator, is shown to be somewhat related to Cramer’s V measure for the 
association between response and auxiliary variables. In fact, the R-indicator is better viewed as a lack of 
association measure since a weaker association implies that there is no evidence that nonresponse has 
affected the composition of the observed data. The theoretical properties of the proposed indicator are 
developed and it is illustrated through empirical studies. 

Finally, in his article, Chauvet addresses the issue of balanced sampling when sizes in each stratum 
are too small for exact balancing. The author proposes an algorithm adapted to the Cube method, 
which guarantees balancing at the population level. A simulation study confirmed that the proposed 
method performed well. 


Harold Mantel, Deputy Editor 
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A new face on two-phase sampling with calibration estimators 


Victor M. Estevao and Carl-Erik Sarndal ! 


Abstract 


This paper provides a framework for estimation by calibration in two-phase sampling designs. This work grew out of the 
continuing development of generalized estimation software at Statistics Canada. An important objective in this development 
is to provide a wide range of options for effective use of auxiliary information in different sampling designs. This objective 
is reflected in the general methodology for two-phase designs presented in this paper. 


We consider the traditional two-phase sampling design. A phase-one sample is drawn from the finite population and then a 
phase-two sample is drawn as a sub-sample of the first. The study variable, whose unknown population total is to be 
estimated, is observed only for the units in the phase-two sample. Arbitrary sampling designs are allowed in each phase of 
sampling. Different types of auxiliary information are identified for the computation of the calibration weights at each 
phase. The auxiliary variables and the study variables can be continuous or categorical. 


The paper contributes to four important areas in the general context of calibration for two-phase designs: 


(1) Three broad types of auxiliary information for two-phase designs are identified and used in the estimation. The 
information is incorporated into the weights in two steps: a phase-one calibration and a phase-two calibration. We 
discuss the composition of the appropriate auxiliary vectors for each step, and use a linearization method to arrive at the 
residuals that determine the asymptotic variance of the calibration estimator. 

(2) We examine the effect of alternative choices of starting weights for the calibration. The two “natural” choices for the 
starting weights generally produce slightly different estimators. However, under certain conditions, these two estimators 
have the same asymptotic variance. 

(3) We re-examine variance estimation for the two-phase calibration estimator. A new procedure is proposed that can 
improve significantly on the usual technique of conditioning on the phase-one sample. A simulation in section 10 serves 
to validate the advantage of this new method. 


(4) We compare the calibration approach with the traditional model-assisted regression technique which uses a linear 
regression fit at two levels. We show that the model-assisted estimator has properties similar to a two-phase calibration 
estimator. 


Key Words: Auxiliary information; Two-phase regression estimator; Starting weights; Separate residual variance 
estimator; Combined residual variance estimator. 


1. Introduction 


levels: At the population level, the total ),x,, 1s known, 


The term double sampling refers to sampling designs 
whose common feature is a selection of two probability 
samples, denoted s, and s, both of them subsets of the 
finite population of interest, given by U = {],..., k, ..., 
N}. The sample s, is realized and observed prior to s. A 
typical study variable is denoted by y; its value y, is 
obtained only for the units k € s. The objective is to 
estimate the population y-total Y = Sy, (if A isa set 
of units, A c U, then we write >, as a short form for 
D,<4 When there is no ambiguity). 

Hidiroglou (2001) discusses two types of double 
sampling, nested and non-nested. This paper focuses on the 
nested type, usually referred to as two-phase sampling: The 
phase-two sample s is a sub-sample from the phase-one 
sample s, drawn from U, sos cs, CU. 

Estimation for two-phase sampling has been examined in 
several earlier papers in a context where two kinds of 
auxiliary information are recognized and addressed by their 


where x,, is a vector known for every & € s,; therefore, it 
is also known for every k € s. At the level of the first 
sample, the vector value x,, is observed for every k € 5,, 
and is thereby known for every k € s; the total },x,, is 
unknown but can be estimated without design bias at the s, - 
level. Two arguments are found in the literature for 
incorporating these two types of auxiliary information in 
estimating Y = Dy y,: the regression fit argument and the 
calibration argument. Under certain conditions they can lead 
to identical estimators, but this is not so in general. 

The regression fit argument prevails in Samdal and 
Swensson (1987), Samdal, Swensson and Wretman (1992), 
Sitter (1997), Hidiroglou and Samdal (1998), Axelson 
(1998) and Hidiroglou, Rao and Haziza (2006). The 
calibration approach in Deville and Samdal (1992) was 
applied to two-phase sampling by Dupont (1995). She 
compares the resulting calibration estimators with those 
obtained from the regression approach. For the same 
auxiliary information, the two approaches may not give 
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identical estimators, although in practice the difference is 
likely to be of little consequence. Resampling for two-phase 
variance estimation is considered in Kott and Stukel (1997). 
Estevao and Samdal (2002) focus on the calibration 
argument and distinguish ten different ways to use all or part 
of the information available at the two levels. The present 
paper also focuses on the calibration approach. It extends 
earlier work by recognizing three (rather than two) types of 
auxiliary information, each having different characteristics. 

In the regression approach, it is natural to fit two linear 
least squares regressions. One set of regression-predicted 
y - values are produced for k € s, using both x,, and x,, 
as predictors; another set is produced for k € s, using only 
the vector x,, as predictor. Both sets of predicted y - 
values, as well as the known total >, x,,, are used to build 
the regression-type estimator of Y, in the manner described 
in section 9. 

The calibration approach is motivated by two factors: To 
create a set of weights that are consistent with known or 
estimated totals for the auxiliary variables and to reduce the 
variance of the estimates made for the study variable(s). We 
want the weights w, in Y¥,,=>,w,y, to achieve 
consistency with the total >, x,, known at the level of the 
population and/or with an (approximately) unbiased 
estimate, made at the level of the phase-one sample, of the 
unknown >), x,,. Since y is observed only at the ultimate 
level (the phase-two sample), consistency “at higher levels” 
on important auxiliary variables will often significantly 
reduce the variance of Y,, = %, w,y,. We can distinguish 
two steps in the process leading to the weights w,, a 
phase-one calibration and a phase-two calibration. 

The two-phase sampling design is as follows: From the 
finite population of units U = {1, 2, ... k, ... N} we 
select a phase-one sample s,. The known positive inclusion 
probability of unit k is nm, =Pr(kes,), and the 
phase-one design weight is a,, = 1/7,,. Certain variables 
may be observed for the units k € s,. Then, conditionally 
on s,, we select a phase-two sample s from s,. The known 
and positive conditional inclusion probability of k is 
T,, =Pr(kes|s,) for kes,, and the conditional 
phase-two design weight is a,, =1/7,,. (to keep the 
notation simple, we use 7,, and a,, rather than the more 
suggestive 1,,,, and a,),; it should be kept in mind that 
both 7,, and a,, are conditional on the phase-one sample 
s,). The combined or double-expansion design weight is 
a, = 4,@, for k es. The analysis of the estimators in 
this article is design based. The term “(approximately) 
unbiased” means “(approximately) design unbiased.” We 
assume mild conditions on the population and the two 
sampling designs, permitting us to discard lower order terms 
in the analysis of our estimators when the expected sizes of 
the phase-one and phase-two samples are sufficiently large. 
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The double-expansion estimator },a, y, 1s unbiased for 
Y = Yy y,. We can produce more efficient estimators by 
taking into account the available auxiliary information. 
Three types or sets of auxiliary variables (called x-variables) 
can be distinguished for two-phase sampling designs. These 
are denoted by x°, %' and X°. Their information 
characteristics are specified in the following table. 


Table 1.1 
Sets of auxiliary variables for calibration in two-phase sampling 
Set of Auxiliary Unit variable Unit variable 
auxiliary variable values for values for 
variables _ total over U kes, kes 
x* known known known 
x known unknown known 
as unknown known known 


Each set may contain any number of x-variables. The 
three sets are mutually exclusive. The properties in the last 
three columns apply to every x-variable in the correspon- 
ding set. All x-variables used for calibration belong to one of 
these three sets. 


2. Phase-one calibration 


For the phase-one calibration, we use a vector x,, of 
auxiliary variables selected from the set X°. While it is 
natural to let x,, consist of all the variables in ¥°, the 
general presentation here allows us to define x,, to include 
some or even none of the variables in ¥°. The phase-one 
calibration weights w,, are derived by modifying the 
phase-one starting weights a,, subject to the calibration 
constraint >’, w,%,, = Ly X,- In our formulation, the 
calibration weights are given for k € s, as 


Wa VG a |! Nc X,)'(d 4), 2.X1p ) a (2.1) 
WiNeTe pe Na =X, aN Ds, %jX, and z, is an 
instrumental vector of the same dimension as x,,. It 
replaces x,,/o;, in the form of the model-assisted estimator 
described by Sémdal, Swensson, Wretman (1992), and 
permits a more general specification of the calibration 
weights. The use of an instrumental vector is discussed in 
Estevao and Saérndal (2000) and Deville (2002). Here and in 
the following, we always assume the invertibility of 
matrices such as the one over s, in (2.1) and those (over s 
and U ) appearing later. 


3. Phase-two calibration 


We use a vector x, of auxiliary variables to produce a 
set of phase-two (or final) calibration weights w,. They are 
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used to calculate Y,, =,w,y, as our estimator of 
Y= Duy. The vector x, = (Xs Xk)» Xiqay) has 
three components, as described below. No auxiliary variable 
can appear in more than one of the three vector components. 
These three components have different roles in the setup of 
the phase-two calibration equation >, w,x, = X and in the 
determination of the phase-two calibration weights. 

The variables in the vector x,,,. are selected from 
among those in the set ¥° U X*. This means that the total 
Xu X41) 18 known and can be included in X. Variables in 
X,, are allowed to reoccur in x,,,, and this is usually 
preferable in order to reduce the variance of the estimator. 
We can specify X,,,. = X,,, but our framework permits 
X,,,, to include variables from x". This allows us to use 
variables with known population totals in situations where 
the variables are too expensive to collect for a large 
phase-one sample s, but are observable for the smaller 
phase-two sample s. These variables are excluded from the 
phase-one calibration because they are unavailable for 
heise 

The variables in X,,,) and x,,,) are selected from 
among those in the set X®° U X¥' U X° provided they are 
not already included in x,,,. The variables in x,,,,) are 
those for which we want to satisfy the phase-two calibration 
equation 2D, W,Xxiw) = Ls, MizXicw)» Where the nght-hand 
side is approximately unbiased for )y X;,,,,). The variables 
iN X,(,) are those for which we want to satisfy the 
phase-two calibration equation 2. WXi(4) = Ls, xX x(a): 
Here, the right-hand side is unbiased for )iyX;,(,). The 
inclusion of both x,,,,, and X,,,) in the definition of x, 
allows us to calibrate on one or both of these vectors and 
provides a general framework for producing different 
estimators from the phase-two calibration. 

The phase-two calibration equation is >, w,x, = X, 
where X is the stacked auxiliary vector 


ys Xk) 
Dy Wi Xkcw) 
Daye A,X Ka) 


A specific variable can only occur once in x. 
Otherwise, the calibration equation may be inconsistent and 
admit no solution. 

The starting weights for the phase-two calibration are 
denoted by a, for kes. There is more than one 
reasonable choice for the a,;. We consider two alternatives, 
both of which seem natural: (1) a, = a, = a, a5,, and (2) 
a, =W,4,, Where w, is the phase-one calibration 
weight given by (2.1). 


(3.1) 


5 


Given the starting weights a;, we determine final weights 
w, Subject to the calibration equation >, w,x, = X. These 
final weights are given for k € s by 

Wir =oa, {1 +(X - x) (>a; Za x, } ‘ts (2) 
where X = >,a;x, is an unbiased or approximately 
unbiased estimator of X, depending on the composition of 
x,. The instrumental variable z, has the same dimension 
as x,. The vectors z,, and z, are assumed to be fixed 
functions of x,, and x,. How to choose z,, and z, is a 
topic we leave for others to address. 


4. Comparison of two options for 
the starting weights 


The objective in this section is to analyze how the final 
weights w, in Ys p = Ls Wy, depend on the specification 
of the starting weights a, in (3.2). We consider two distinct 
cases based on whether or not the auxiliary variables x, are 
used for the phase-two calibration. When we carry out the 
phase-two calibration, the two different choices for starting 
weights generally lead to different estimators. We show that 
these estimators are asymptotically equivalent under certain 
conditions, commonly found in practice. When we have no 
phase-two calibration, the two choices for starting weights 
lead to two other estimators that are usually less efficient 
than those obtained by performing the phase-two 
calibration. 


4.1 Estimators with phase-two calibration (x, # ) 


As noted Per woushy, there are two alternatives for the 
Siarinie weights 7," in(5.2):¢b) a) = ay ta) A, and (2) 
a, = W,@,, Where w, is the phase-one calibration 
weight given by (2.1). We now provide a detailed analysis 
of the form of the estimator under these two choices. In this 
subsection, we look at the more interesting case where we 
perform the phase-two calibration (x, # @). In the next 
subsection, we consider what happens when we do not carry 
out the phase-two calibration (x, = @). 

Our procedure is as follows. First, we derive the 
linearized (asymptotic) form of Y, p based on the general 
starting weights a,;. Then we substitute the two choices for 
a, in this expression. We determine Y,» based on the 
starting weights a, = 4, =a, 4,. We denote this 
estimator by Y,,,, and derive its linearized form, ¥, na? 
Similarly, we obtain y, p based on the starting weights 
a, = W,,a),. We refer to this estimator as Y,,,, and derive 
its linearized form, Y,,,,,,,- These two forms are slightly 
different but we prove in Result 4.2 that 1 Destine ‘A EN. 
under certain conditions. 
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na 


We start by inserting the weights w, into Y,, = 


x; y, and wniting the estimator as 


na 


= f 
Yyp ms Deu Xe By; no as oe Wie XW) Boy, nxn) 


f * 
i pee 44 X4(a)Byy; ya) + Dus % Sos 0k 


* 


uF (X ad X)'(Byy:x) a By...) (4.1) 
where Laelia = (wae a, Zy kt Ls a, Zi Vio By. x) 
Qu Z,X; edt, Zi Vk and Bi.» =(Bi.»0 Bo:n092 Boxx@) 
is the partitioning corresponding to xX, = (Xj. Xgwys 
Xi(q)); Our subscript notation of the form (v,; v,) 
identifies the variables in the regression. The term v, refers 
to the independent variables and v, identifies the dependent 
variable or variables. For simplicity, the instrumental 
vectors Z,, and z, are not included in the notation. 

The term €,. 4 = Ye — X,By,, x) 18 defined for k € U. 
Note that although e@,,,. ,), looks like a regression residual, it 
does not arise as the result of fitting a proper regression 
model. We then develop the term 2. Wy Xi(»)Biy. on) 10 
(4.1) by inserting expression (2.1) for w,, and making use 
of the phase-one calibration equation }, w,X\, = Lu X,x- 
We obtain 


! — 
pss WX) Boy; nw) = 
, 
X, Bus: x,) 7 aK Ai. E(xB.,,): x, )k 


+(X, -— X,)’ LB sn.) Bape) at4-2) 
n tes f y-1 ’ 
where Bus. :x,) =(Ls5, ZX) Vs, FZ X40n)Bey-xyw) 
converges in probability to (and is approximately unbiased 
= Poni ’ 
for) Bop. :x,) = Qu Ze Xe) Lu Ze XeowyBey; nowy» add 
yet ’ eee) 5 
&(xB,,.);%, arm Xx (Ww) Bey:xw) X1.B os. :x,) is defined for 
kKeU 
We can interpret €.5 .x,)x aS a residual arising from a 
population fit based on a generalized regression of 
Xi(»)Byy: x(w) a8 the dependent variable and x,, as the 
predictor vector. Replacing expression (4.2) into expression 
(4.1) for Y,, leads to 


A 


ie ’ ’ 
Y,p = doy By. an + X, Bos.:x,)) 
' 
re DE, Ay (X(4)B iy; x(a) * CB: x)k) 


* 
+ Daa Ey; x)k 


+ (X,— X) (Bas: 7 Bas,,:x,)) 


*(X — X)'(Bi,.y - favo) (4.3) 
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The following result establishes the relationship between 
the estimators obtained for the two choices of starting 
weights. 


A 


Result 4.1: The linearized forms of Y,,, and Y,,,, are 
related” ‘by ‘the? ¢quation” “Yy55 5 —Y>pua 
(X, — X, CB Pa Pua: 


alin 


Proof 


We consider expression (4.3) under the two possible 
choices for a;. First, with a; =a, = a,a,, we obtain 


Y,p, given by 
5 ’ ’ 
Yopa. = Dy SBoixxn + Xx Bos.x,)) 
/ 
4 me (Xia) By:xyay) + %xB,.):x,)k) 


+ DG C(y:x)k 


+(X, - X,)’ (Bas,.,:x,) - Bius.,:x,)) 


+(X — X)(Byy - By.x)- (4.4) 


The term Bas = (Dy a,2Z,x,) | Ds a,Z,y, converges 
in probability to (and is approximately unbiased for) 
By.) = (Lu ZX) OS Pr. yw het tirst term is constant 
and does not contribute to the variance of Y,,,. The next 
two terms are random quantities, defined as sums over 5, 
and s respectively. The last two terms are products of 
differences with zero or almost zero expectation. As for the 
product (X, — X,)’ (heey Bis,.,;x,))» both differ- 
ences are functions of the phase-one sample s,. We know 
that X, is unbiased for X, and Hw (es is approximately 
unbiased for B 38, ;3,) Under fairly general conditions, 
NX, -X,)' Bas.) 7 Bas.y:n)) = Op(m ), where 
n, is the expected size of s,, assumed sufficiently large. By 
a similar reasoning, N '(X — X)'(By.. - By,..)) = 
O.(ay where 7 is the expected size of s, also assumed 
sufficiently large. Consequently, we can drop the last two 
terms of (4.4), because they are of lower order than the 
preceding terms: N'Y. ay (Xia) Byy-x) (a + €(xB,:x)k) 3S 
Op(a?)y and NO Zeagegey isO, Ga) Whe tirst 
three terms define the linearized form of Y,,,, 


Ypalin = De boas + Xi, ee 
J 
+ aa a, (Xj (a) By; xya) a (xB, 1k) 


+ > aes (4.5) 


Now let us consider expression (4.3) under the second 
choice, a; = Ww, @,. This leads to Y,,,, given by 
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Yipw = Dy KoBo, a + Xe Bos.) 
oe A 14 (Xia) Boy; xa) + SaB,.,;%,)4) 
Feo Geixhk 
+(X, -X&)'(D,, Ay Zp Xy \e Dd e248 x)k 
POaHAX) (Be,.) — Bos) (4.6) 
where By, — = (Ls Wp ZpXp) Lis Were Ze Vs and 


X = Y, w,@,X,. The first three terms of Tp aeaie athic 
same as those found in expression (4.4) for Y,,,. The 
fourth and fifth terms differ from their counterparts in (4.4). 
Although 3) le and X are functions of the phase-one 
calibration weights w,,, we do not need to replace them in 
By, x) and X in the fifth term; this would simply split the 
lower order term (X —- Xx)’ (BY. 7) 7 By) mito” other 
lower order terms. Therefore, we can drop the fifth term of 
(4.6) when the sample sizes are sufficiently large. The 
fourth term can be written as follows. 


sf 
f , 
) var 444.241; | SSE Ae as 


= (X, — X,)'(B,,. x,) =B. Po.) 
a (X, aa X, )( B,,. xX, ) 5; By xX, y) 
ae (X, 4 X,)'( B,. x,) =BEs BGS 


= 


+ (X, - X)'(D, 44, 2,X1, | 


Pe AZ y; xk ~ Dare NZ iE y: xk Hh (4.7) 


The quantities in this on are defined as follows: 
Ba oa, A ZX) x. yy Zi Xp and Besa 
(Ds, ERS D's, 41nZ1,Yx- The statistic By,.,)s, can not 
be computed from the phase-one sample because the values 
y, are only known for k € s. It is implicitly defined for 
the purpose of determining the linearized form. We can 
define such a construct in the same manner as By isa 
function of the unknown quantity B,,. .\W,- 
B,,.x,)s, 18 approximately unbiased for its corresponding 
population quantity Bi...) = (Lu 2 See aces 
Similarly, B,,.,,) 18 approximately unbiased for B 
(Xu Z,%1,) | Xu Z,X;- As before, we can argue that the 
last three terms of (4.7) are of lower order than the first term 
(X, — X,)(By,: ) Be: x)By, 9)» which provides the 
linear approximation. The substitution of this term into (4.6) 
leads to the linearized form of Y,>,,, 


Buys %) 
Now 


(x; x, ) 7 


(x; x) 


v4 
Puss) = DB Brae + Xi, Bi. :x,)) 
» dine Ay (Xi(a) Boy; 2X0) CuB,:x,)k) 
is Dis Myx 
ot Os Ce xB x) —B,,. A ereee (4.8) 


_ Comparing (4.5) with (4.8), we see that Veet: 
Yypain + (X— X,)(By..,) — Bu: x)By,.)) aS stated in 
the result. This completes the proof of remit 4.1. 

Result 4.1 shows that in general, the linearized forms of 
Y,>, and Y,,, are not the same. However, they are the 
same under certain conditions. Let us consider the case of 
nested calibration (not to be confused with nested 
sampling), meaning that x, includes x,,. Then x, is of the 
form x, = (X;,, x,,)' where the vector x,, is composed 
of the remaining variables. We now state and prove the 
following result. 

Resuit A It X= (Xk, eau Ze (Zizek) tne 
Yop wiin =Yo atin and Yon, .aNde Yo... ate 
asymptotically equivalent. 


Proof 


The proof follows from result 4.1 by showing 
Bey ob B,,.,,)= 0 under the specified conditions. 
We have 


(x; x)" (3 x) 


B 


(a) B 


=(y Zip Xi ) i pale Dy ) 


=, (eZ X12, Wp), Sitice 
z',)', it follows 


(GX, B (y3x) 


where  h, = y, 
Yu Zh, = 0 and we assume z, = (Z,,, 
Yet, =. 0 and By oBat By... = 0, Therefore 
from result 4.1, Nay in = Yopatin- Since their linear 
forms are the same, Y,,, and Y,,,, are asymptotically 
equivalent estimators. 

Interestingly, Result 4.2 only requires that we include 
X,, somewhere within x,. Obviously, it makes sense to 
include x,, within the component x,,,. of x, because the 
X,-totals are known. However, we obtain the same 
asymptotic result as long as all variables in x,, are included 
somewhere in X, = (Xpiys Xen) Xia) ‘IM practice, we 
often find x,, =X, with z, =x, and z, =x, = 
(x;,, X,,) Where x,, is the vector for the remaining 
variables X,,,,. and Me (a: This satisfies the requirements 
for the asymptotic equivalence of Y,,, and Y;p.,,. 

To study the properties of Y,,, and Y,p,, we work with 
their linearized forms given respectively by (4.5) and (4.8). 
With appropriate definitions for the residuals é,, €,, and 
€,,, We can represent Yopcin and Yypiyy, as the sum of 
three terms: a constant term >} e,, a phase-one expansion 
term >, 4,,€,, and a double-expansion term }’, 4,€>,, 
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Yyptin = Doe 7 Yak Np ee + eG: €,,- (4.9) 


This makes (4.9) a suitable starting point for studying the 
bias and the asymptotic variance of the two estimators Y,p , 
and Y,>,,. 

For the linearized form Y,,,,,, given by (4.5), the three 
residual quantities are defined as follows for k € U: 


' ' 
Con = Xie) Bysxyn + Xe Bos,,,:3) 
hes ! ' 
Qe = Xea)Bor xa) + Xen Bos ayn 
—-x,_B 
1k" (xB...) ; %)) 
/ 
oo) ‘oh beilaae > Vinbliegl 25) Bi, x1) a X iw) By. xyw) 
’ 
a Xi(a) By; xa) (4.10) 


Note that e,, is simply e,.,),- Similarly, for ae 
given by (4.8), the residuals have the following definitions 
fork € U: 


a ' 
Co. = Xe) By. 


Xi) 3 B,,. xe Be Ban) 


, / 
Ce = Xia) Boy; way + Xen) Bo; 0) 


) f 
x Xi (Bos..:x,) . B.,. x) ~ Ba, «By, ») 
pay , 
pee Verwan) Bi. x) 
f , 
— Xi Bo; non) ~ XecayByy; xa): (4.11) 


Note that in both cases, e, + e, + €, = y, for every 
k, and hence Yy(e, + @, + @,) = Ly yy, = Y. This 
additivity allows us to prove in section 5 that Y,,, and 
Y,»,, ate approximately unbiased. To save space, we 
concentrate on the properties of y pq im the remaining 
sections. However, the analysis is similar for Y,,, and the 
method for variance estimation proposed in section 7 can 
also be used for this estimator. 


4.2 Estimators without the phase-two calibration 
(x, > ?) 


If there is no phase-two calibration (x, = ¢), then 
w, = a,. Accordingly, the final weights are either w, = 
a, =a, a, OF W, = WwW, a,. The first alternative gives 
the double-expansion estimator >, a, y,. The second 
produces a different estimator that is usually more efficient. 
However, both of these are generally inefficient compared 
to the estimators obtained by carrying out the phase-two 
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calibration. The linearized form of the two-phase estimator 
with w, = w, a), is obtained by wntting it as follows. 


t a ’ v7! 
Ye 3 Beas XiBiy.x,) + see 


+(X, -.X,) B - Be.) 


(y; x, )S; 
A =i 
Uy 
+X — X)(D, 2X) 


(do. eZ =" pat NZ Vi ) 


The terms Beene and By,.,, were defined in the 
previous section. When the samples are sufficiently large, 
(Xx ee 424%) (Ls ZuPe — Xs, M21. ¥_) and 
(X, -— X,)' (By..,)s, — By.x)) are of lower order and 
can be ignored. This leads to the linearized form of this 
estimator. 


(4.12) 


A aA 


= X'B -oX, Big synt ara yg (Aus) 


Ys ptin VessGys Xp) 


We can also write this linearized form as a sum (4.9) of 
three residual terms, with the residuals e,, e, and e,, 
having following definitions for k € U. 


fas ' 
Con = Mg Bux) 


pee 
Cm > x1, By,.x,) 


Cnn = Ve (4.14) 


These residuals show a resemblance to those given by 
(4.10) if we set x, = ¢ and remove B,,.,.. Note how 
B,,.,,) has the same role as Bus,.,: x) in (4.10). As before, 
ey + 6, Pie = yyertor every *kPandiiencelyy, (2, es 
Gi, Pye = LoVe =. 

The double-expansion estimator is a special case of this 
estimator when we also have x,, = ¢. This means that 
B.,.,,) 18 not defined. The corresponding definitions for 
€5,> &, and e,, are simply e, =0,e, =0 and e,, = 
y, for k € U. 

In the following sections, we examine the bias and 
variance of the two-phase calibration estimator ie pq and 
we propose a new method for estimation of variance. We 
can derive corresponding results when there is no phase-two 
calibration because the residuals for these two groups of 
estimators have similar properties and linearized form. The 
only difference occurs in the estimation of variance. We use 
the same variance estimator (as described in section 7) but 
the residuals are estimated by using é, = —x1,B,,. a. 
where Boy. .y)s = (Qs & Zu Xi Je ua eel ey ale 
Cn = Vee 
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5. Bias and variance of the two-phase 
calibration estimator Y,,, 


The two-phase calibration estimator Y,,, = Y.w, y, 18 
approximately unbiased for Y = >, y,. To show this, we 
derive the expectation of the linearized form given by (4.9) 
via the usual method of conditioning on the phase-one 
sample s, We have E(d.4,€,)=E, Ess (Ls er) = 
Es, (Zs, Mere) = Ly Cre Es, Ls, nee) 3) Cy, and Ly Coy 
1S a constant term, so 


E(Yyp atin) eo 

This shows that Y, palin IS unbiased for Y. By (4.4), 
Pars = Yad. +R... So ithe (bias of om equals the 
expectation of R, which is the sum of the two lower order 
terms (X, — X,)’ (Bis, eh Bis. gy )) and) (X= XX)’ 
(Be. on B ee As pointed out in section 4, each of these 
terms has expectation close to zero. It follows that Y,,, 
approximately unbiased for Y. 

The variance of Vase = Ysw,y, 18 closely approxi- 
mated by the variance of the linearized form Y,p,,, given 
by (4.9) with residuals defined by (4.10). Its first term, 
Xu o, 18 constant and does not contribute to the variance. 
Therefore, 


= Dolce + &, + €) = 


ne) ae Ay, ey, + pas a, ex): (5.1) 

We use (5.1) as the starting point for deriving a variance 
estimator for Y,,, ;,,- Two different approaches can be used 
and it is of interest to compare them. The one in section 7 is 
new and more interesting because it produces a more 
efficient variance estimator than the one in section 8, 
derived by the traditional technique of conditioning on the 
phase-one sample s,. The residuals e, and e,, given by 
(4.10) play an important role in both derivations. 


6. Preliminaries for variance estimation 


Our objective is to estimate the variance V(Ysp in) 
given by (5.1). This is done in sections 7 and 8 by two 
different arguments. The residuals e,, and e,, are defined 
for all k © U but they can not be computed. They must be 
replaced by estimates é,, and é,,. These estimates, formed 
in the image of (4.10) are 


A 


A SORE . , 
Ex = Xpa)Byy. aya) + XecwyB 


(y; xXw) 
~ SF ara for k € s, 
ey = Ve x, By.» 
ae 4 a Xi) Eh Ti Rr HE GEnOD 


- <b for kes (6.1) 


9 
where 
z =j 
wear 
Bt oue (Daas Z; x) Daeg) ZV 
ay ’ m! y 
ay (Bie SB x)(w)? » By. x(a) 
and (6.2) 
(Bas tie 
(Day Ay, Ziz tA man Ay, Z yx IEE: x)(w)* 
The term B. in the seunuen of é, is the esti- 
Bow) n 
mate of ae “xia =(XyZ, Xe ean Xi Boexvwy't 10 


(4.10). Two replacements are required in B..p |. x, to arrive 
at Baas >) First, sums over U are replaced by ap- 
propriately weighted sums over 5,, giving Ble) = 
(2s, Mt vii eed i Ds, Mk Zix XicwyByy:xyw) ‘In this expres- 
sion, B30) is still unknown, so we replace it by its esti- 
mate Bi, »») to arrive at Baga 

A key point to note is that estimates é, can be obtained 
for k © s,, because X;,(,), X,¢,) and x,, are all known for 
k e€ S,, but estimates é€,, can only be made for k € s, 
because y, is available only for k € s. The fact that the 
estimates é, are available for k © s, rather than k € s 
allows us to construct (in section 7) a more efficient 
estimator of V(Y,,,,,,) than the traditional approach to 
variance estimation (in section 8) where all estimated 
residuals are calculated only for k ¢€ s. 

The design weights a, = 1/n,,,@, = 1/n,, and 
a, = @,@, were defined in section 1. In the following 
sections, we also need the quantities given below, defined as 
functions of the second-order inclusion probabilities 
Taper (Ke de Ces, reall My. = PECK OCLES |S, ): 


pe = lM e, Aye = 1W/TMygy, Age = Age Bee 


Dipe = Ape — Apes Doge = Ay44q¢ — Aquos 


Dy = AA, — Ay 


Here, m,,, and a,,, are conditional on the sample 5. 
All first-order and second-order inclusion probabilities are 
assumed positive. Using this notation and the above results, 
we now develop two different variance estimators in the 
next two sections. 


7. The separate residual variance estimator 


The variance of VB Palin 1S given by (5.1), where e,, and 
e,, are defined by (4.10). It can be expanded as 
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Ve Ae = alee Ay, ex ) Li VAS é,) 


+2 Cov()), ip Gin» Dee) en): 


If we knew the residuals e,, and e,,, unbiased estimates 
for these three components would be given respectively by 


Le les, Die Ee “in 


er. D,, 2% ©29 


- ee bes Dik? Ay, C1, C55: 


(7.1) 


(7.2) 


The proof of unbiasedness is similar for all three 
components. For example, for the second one, we have 


E, END ee, jasD nee) 
= E, Oiyus ps (Dy, /aty4)€r4&2, | 
i DE Dea ees 


2 
= > eA li Qelaplehens 7 (Doane) 


= #[(Z.eeu) | - [(E.aen)] 
= Vass ). 


We now replace the unknown residuals in (7.2) by the 
respective estimates given by (6.1); that is, e, by é, for 
k es, and e,, by é, for k e€ s. Then, the resulting 
three components are added to arrive at the “separate 
residual” variance estimator 


A An 


Vie Yop atin) oo Pl res, Pike ie ee 


i De phat Cn, Ere 


+ 2 Dia res Pine Fre Erne re- (73) 


The term “separate residual” and the corresponding 
subscript sr reflect the fact that (7.3) keeps the residuals 
separate, where é,, is defined over the larger sample s, and 
é,, over the smaller sample s. The fact that residuals 
computed for the larger sample s, can be advantageous for 
variance estimation was recognized by Axelson (1998). 
However, his derivation differs from our calibration 
approach based on x,, and x,. The technique for variance 
estimation of the two-phase regression estimator in 
Hidiroglou, Rao and Haziza (2006) has certain traits in 
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common with our approach, but there are also considerable 
differences. 


8. The combined residual variance estimator 


We arrived at (7.3) by recognizing that the estimates é, 
are obtainable for k e€ s, The traditional approach, 
reviewed in this section, is to derive a vanance estimator by 
conditioning on the phase-one sample s,. This produces a 
variance estimator where all required residuals are defined 
for k € s. Later, we compare it with the more efficient 
(7.3). From (5.1), we condition on the phase-one sample s, 
to obtain 


A 


V(Y. 


peep yo el See ae Bare Ape, +) %22¢ | 
+, Ea ae (oe Aye, +), %&x 
s, Dan Ap, pai 1,4 ) 

5," s|s, (yearer| 


= Se a4, 2:24 )+ Es, Yoys, (DS 4.ae) (8.1) 


where @,, = @&, + @, 1S called the combined residual. 
From (4.10), we obtain the following. 


, , 
Cae Nia a Sie OG son X1,Bos.,.:x,) 
, , 
Cn = Ve ~ Xe Bory 7 Xeon Boyan) 
, 
— X (a) Boxya): (8.2) 


It is straightforward to define estimators of the two 
components V, (2's, 4, @2,) amd E, Vs. (Qs a &,)- 
Each of these has the form of a double sum over s because 
€, and e, contain y, which is only available for 
k es. The first component uses @,, = 6, + €, = 
: for k e€ s. We then 

as an estimator of 


Ye — XeBos ayy ~ XuBos,, x) 
have Yes Lees Dike Gree rn e120 
Vs (Xs, Mx 12% )- 

For the second component, we use the residual estimates 
Ce Via x Be given by (6.1) for k e€ s, and 
obtain Diges Lres Lp fap 1 Crp... dS2am .estimatorsot 
E. Vsis (Ls, )- Summing the two estimated terms we 
have the following variance estimator, where the subscript 
cr indicates ““combined residual”, 


Vie (Yop atin) = ee pes Prk e Fone Erne E20 


+ ye pes oke Dy By p @ 24 &29- 


(8.3) 
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Let us review how (7.3) and (8.3) differ. The separate 
residual variance estimator (7.3) starts with the expansion 
V (Vy pain) = KOs Ap Oy )+V (Ys ay Cyy + 2Cov(d's, A, igs 
ds &,). We estimate these three components separately 
as functions of the residuals e, and e,,. The resulting 
variance expression has three terms: a double sum over s, 
in terms of e,, and e,,, a double sum over s in terms of 
€,, and e,,, and a cross-sum over s, and s in terms of 
é, € 5, and e,, € s. Finally, we arrive at (7.3) by 
estimating e, by é&, for k € s, and e,, by é, for 
Kes. 

The combined residual variance estimator (8.3) arises 
from the traditional ee on the phase-one sample 
5, a VYypain) = Vs Ess, Gsema) + E,V. Vs (agin): 
This leads us to ers é,, and e,, as ae = \Crpehie yp 
in the first term. The second term, E, V,., (Vepipen)attisa 
function of e,,. Since e,,, and e,, can only be estimated 
over s, the resulting variance estimator becomes a sum of 
two terms, each of them expressed as a double sum over s. 

The separate residual estimator (7.3) is more efficient 
than the combined residual alternative (8.3), because it is 
based on residuals €, obtained for the typically larger 
sample s, The advantage of (7.3) over (8.3) is illustrated 
by the simulation in section 10. The approach behind the 
separate residual variance estimator (7.3) can be extended to 
three-phase sampling and other complex designs. In those 
extensions of the technique, we proceed in a similar manner, 
starting by a derivation of the linearized form through an 
expansion of the variance components and the determina- 
tion of the appropriate residuals. 


9. A comparison with the two-phase 
regression estimator 


Samdal, Swensson and Wretman (1992) developed a 
two-phase regression estimator for Y = Dy y,, based on 
an earlier paper by Samdal and Swensson (1989). It is 
useful to see how this estimator, denoted here by i 
compares with the calibration estimator Y,, considered in 
the preceding sections of this paper. When based on the 
same auxiliary information, the two estimators are “close” 
but not identical. This is because the estimator Y,, is 
derived by calibration in each of the two phases, whereas 
the two-phase regression estimator Yee is derived by 
model-assisted reasoning. 

We now describe the two-phase regression estimator of 
Sarmndal, Swensson and Wretman (1992). Their derivation 
involves the fit of two linear regression models with the use 
of the available auxiliary data; one at “‘the top level” and the 
other at “the bottom level”. These authors develop a 
corresponding estimator of variance, via the traditional 
conditioning argument. We compare their variance 
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estimator with the combined residual variance estimator 
(8.3), also developed by the conditioning argument. The two 
variance estimators do not agree exactly, because the point 
estimators are slightly different, but they are numerically 
close, as shown in this section. 

Let x,, be a vector of auxiliary variables with known 
population totals, and let x, = (x},,x;5,)', where both x,, 
and x,, are known vector values for k € s,. The total 
~uX,, 1S assumed known whereas the total },x,, 1s 
unknown. The predicted values produced for k € s, by the 
two regressions fitted at the “top level” and “bottom level” 
are given respectively by 


A 


Dix = XB, 
with (9.1) 
x Ly 
B,, = (rapa tteicia) eds Sicha ie | 
and 
Yd, re] x, B, 
with (9.2) 


. ! 2 i: 2 
B,.= Dag mites! a Gylioms 
The resulting two-phase regression estimator Y,,, of 


ie 7 (Daal Ba. a: Ds He Oe * Vix) 


+ > a,(% — I): (9.3) 


Can - be interpreted as a calibration estimator? To 
answer this question, let us determine the implicit weights in 
(9.3). We can write lei >~;W,¥,, With weights w, 
identified by substituting (9.1) and (9.2) into (9.3) and 
simplifying. We find w, = a, g, = a, 4, g,, Where the 
calibration factor g, is givenfor k € s by 


— f 
Bae i+ (Diu%n * 8 a4, X,, ) 
l Di 
le ch sence canab x,,/S%, 
+ (DK a,,X, - Ds HX) 


De. a,X,X (/o2) x, /o7. 


The weights w, are not explicitly stated in Samdal, 
Swensson and Wretman (1992). In what sense, if any, can 
w, be considered a calibration weight? To examine this, we 
first replace y, in (9.3) with x;,. Using (9.1) and (9.2) with 
y, = X, gives Dyx;, as the right-hand side of (9.3). 
Thus, the weights w, = a,g, Satisfy D.wX, = 
Yu X,,- Next we replace y, in (9.3) with x5,, again using 
(9.1) and (9.2) to obtain 


(9.4) 
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pase. Ay, X5, + (Diu *u . yah ayy, )' 
= 
ae sese lesa 
(dere Loren 


Although (9.5) is an approximately unbiased estimate of 
the unknown x,, -total yj x},, it does not have the usual 
form of the right-hand side of a phase-two calibration 
equation, such as }), a,X3, OF Ds, Wy X>4- However, it is 
close. If we replace the two sums over s with appropniately 
weighted sums over s,, then (9.5) becomes 3s Ww, Xo, 
where w,, is given by (2.1) with z,, = x,,/oj,. Thus, the 
implicit weights w, in Ye calibrate exactly on the known 
population x,, -total, and they come close to calibrating | on 
the estimated x,, -total >’, w,, X>,- This suggests that ee 
should have properties similar to an estimator Y,, obtained 
by defining x, in Y,, as x, = (xi,, x},)' with Xi) = 
Xigo Xe) = Xo, and X,,,, = g In addition, the form of 
the model-assisted estimator implies z,, = x,,/6;, and 
Zz, = x,/o;,. Since x, includes x,, it is reasonable to 
define z, ="X,1G7, 45,2, =a Ota, 1G5,)- These 
specifications meet the requirements for asymptotic 
equivalence of Y,,, and Y,,,, so we do not need to worry 
about the choice of starting weights in Y,,. We can simply 
work with re pq as the estimator comparable to Lo Now, 
let us look at variance estimation for Vg and the estimator 
Y,»,, under these specifications. 

The variance estimator of Samdal, Swensson and 
Wretman (1992) contains calibration factors denoted g,, 
and g),,. They are not to be confused with g, given by 
(9.4). If we disregard g,, and g,,,, both of which are near 
one and of limited numerical impact, their variance 
estimator is 


Ae ee i Sipe es Dye Bax Eres Eres 


(9.5) 


P ya pes Pree Mk Ue Les Er (9.6) 
where, for k € s, 
A {) aN 
Cxs = Ve — X,B,, and &, = y, x ABS (7) 


Both components of (9.6) are double sums over s, 
reflecting the fact that both é@,, and é,, can only be 
obtained for k € s. Formula (9.6) looks similar to formula 
(8.3) for the combined residual estimator but how different 
are the residuals in the two formulas? Let us look at the 
residuals for the comparable point estimator. As noted 
above, this estimator Y,, has x, = (x/,, x5,) with Xi) = 
Xie> Xi) = $2, = X,/o, and z, = 
x, /0;, = (Xx|,/0;,,X5,/63,). Under these specifications, 
the residuals é, and é,, in (6.1) are given by 


= Xap Xkca) 
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— x! n 


/ noe Ss 
1B 8,: 0) for 


A 


' 
Meare By. xy) 


~ Us Rites for k € s (9.8) 


n 


eR ap rp! ’ 
where By... = (Bi. x\1)>Bi;x2)) corresponds to the 


partitioning of x, = (x;,, X5,)’ and from (6.2) 


Box (D5 %2e% | (>) Zee) 
2 a 
’ 2 
B 8.x) a py 4X 4X1e/ Ti | 
Dusen XieX B : 
Ciel ae) aad O92) ee 


The residuals é,, in (9.8) are the same as é,, in (9.7). 
But how do the residuals é,, = &, + €,, obtained by 
adding in (9.8), relate to their counterparts é€,,, in (9.7)? To 
find this link, we first show that B,, =(X;a, x, Xi,/O,) 
Ysa X,Y, /O;, can be written as 


(9.9) 


BoB 


(y; x)() 
A é 
' 2 ’ Zs 
+ aerl  8 [oi ) aoe oe x(2)! Fix ). (9.10) 


To see this, we start with B, way 
satisfies D,a,z,y, = (Xsa,zZ,x',)B 
can also be written as >),a,Z Vi = Ys Qy2, (XB aa 
x}, Ba CEs) osinice 2, (mt, (G4), Xap gp) ete page Ss 
of this equation corresponding to x, is 2 si Siloi= 
Ys XX Bey, x)(1) Oi + Ds A XX, By, 3y2)/O1K: Premulti- 
plying both sides by (\, a,X,,X/,/6,,) , we obtain (9.10). 

Then, starting with (9.8) and using the definition of 


which by definition 
ain This" equality 


Be given by (9.9), we have 
(xB,,) x,) 
Crap = Cp + Coy 
— f : ; 
= ete Sy Besa, a xBis Boy: %) 


’ B ’ 5 \o 
Ve > Bx )Poy:xy1y + ake AXXO}, 


1p 2 
(Be AX, Xo4 B,,. Las Kage }. 


In the expression within curly brackets, let us replace the 
two a,,-weighted sums over s, with the corresponding 
a, -weighted sums over s; the result is equal to B,, as 
given by (9.10). This means é,, = y, — xi,B,, = é,.. 
In summary, é,, = &,, for k € s and é@, = é,, for 
k e€ s. Hence, the variance estimator (9.6) for the 
two-phase regression estimator i should be numerically 
close to the combined residual variance estimator (8.3) for 
the calibration estimator Y,, defined in this section. We 
present empirical support for this through the simulation in 
next section. 
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10. Simulation 


In this section we present a small simulation to validate 
the claim that the separate residual variance estimator 
Vay”) given by (7.3) can be considerably more 
efficient than the combined residual variance estimator 
V..(Yspain) given by (8.3), and that the behaviour of the 
latter is very similar to that of the two-phase regression 
estimator V (Yee) given by (9.6). We created a population 
of N = 5,000 units in two steps as follows: First, the 
values (u,,, u,,) for k = 1, 2, ..., 5,000 were generated 
by 5,000 realizations of the independent random variables 
u,, ~ 2Gamma(4) and u,, ~ 3Gamma(6), where the 
Gamma(a) distribution has density f(x) = [I(a@]™ 
x*"!e™ for x > 0. Secondly, the values of the variable of 
Ae es were created as y, = 10+ u, +3u,, + €,, 
k = 1, 2,...5,000, with ¢, ~ 5Normal(0), where 
Normal(0) is the standard Normal distribution with mean 0 
and variance |. The target of estimation in the experiment is 
the population y -total Y=), y, = 358,205. For the phase- 
one calibration, we used the auxiliary vector x,, = (1, u,,)’ 
and z,, =X,,. That is, the weights w,, for k € s, were 
determined by calibration to the known total (NV, Suu) = 
(5,000, 39,611.8). For the phase-two calibration we used 
X= (Kia Xk) Kea) With Xpqy = Ch M4) Xecwy = Moe 
Xia) = @ and z, = x,. These specifications satisfy the 
conditions for asymptotic equivalence between 4 pq and 
Y,p,. Therefore, for this simulation, we can work with 
r,» and its linearized form Vers. 

For each phase-one sample s,, the final weights w, for 
the estimator be Pa = LsW,Y, were determined by 
calibrating to the known totals given by the vector 
(N, Xu Me» Ls, Wiz Max) = (5,000, 39,611.8, Dw, u,). It 
is important to note that it was not necessary to have 
Xi(a) = % in order to run a simulation to compare 


11,000 - 


10,000 ¢ 


SO FS OS io) ere Ti 
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800,000 
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13 
FA am) and V. Pex, ot) However, we can not 
compare VouYiesie) and V( fp) unless we define an 
estimator y, pq comparable to Y,,, and to achieve this we 
need X,(,,) = ¢, as noted in section 9. 

We drew repeated sample pairs (s,, 5), where s, is an 
SRS of n, units from U, and s is an SRS of n units from 
s, Here SRS stands for simple random sampling without 
replacement. We worked with different size combinations 
(n,, n): (4000, 3000), (4000, 2000), (4000, 1000), (3000, 
2000), (3000, 1000) and (2000, 1000). If n = n, two- 
ere sampling is equivalent to one-phase sampling, and 

Pop ain) and V,,(P>p gin) are identical. 

For each éombiiation (n,n), we realized 100,000 
sample pairs (s,, s). Based on the data for each of these 
outcomes, we computed the separate residual variance 
estimator V. ie palin)» the combined residual variance 
estimator V,,.(¥,>,4,) and the variance estimator V( Pe). 
For this purpose, we used the respective expressions that 
follow from (7.3), (8.3) and (9.6) when SRS is specified at 
each phase. To save space, these expressions are not shown 
here. We obtained 100,000 realized values for each of the 
three variance estimators. Figure 10.1 shows the distribu- 
tions of the 100,000 V -values for n, =4,000 and 
n= 2,000. 

The figure shows strikingly different distributions for 
V4...) and. V. Oe ial. The distribution of the 
separate residual estimator V, S08 Pain)? IS much more 
concentrated. Thus V,, (Y, ney is more efficient than 
ae obs palin) and on average, it produces considerably 
shorter confidence intervals. We also note that the 
oueer of V( Ye) is very similar to that of 

V..(Y5p qin): This supports our analysis in section 9. Similar 
results were obtained for the other sample sizes in the 
simulation. 


825,000 


VY.) 


2Pa a) 


Figure 10.1 Distribution of 100,000 realized values for V,,(¥2patin)» Ver(Y2patin) 2d V (Frog) 
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To obtain a measure of the efficiency of the three 
variance estimators, we computed the simulation variance 
of the 100,000 V -values. These simulation variances are 
shown in Table 10.1, Table 10.2 and Table 10.3. The 
numbers are dramatically lower for V,, (% pain) than for the 
other two. Table 10.4 shows the relative advantage of 
ViEUYaw ae) over V.,(¥,p,,,). For this population, the 
simulation variance of VAY Pain) 1S less than half the 
simulation variance of V., (¥5p a tin): 


Table 10.1 
Simulation variance for the separate residual variance 
estimator V,,.(Y2p atin) 


n 


n 3,000 2,000 1,000 
4,000 64.82 95.91 484.92 
3,000 1179162 1,806.79 
2,000 13,995.94 


Note: Actual values are the displayed values times 10°. 


Table 10.2 
Simulation variance for the combined residual variance 
estimator V..(¥3p atin) 


n 
n 3,000 2,000 1,000 
4,000 153522 364.08 1,290.41 
3,000 2,449.05 6,855.69 
gli DOO 2A haute wmeewiipe re” beiga 99220588 
Note: Actual values are the displayed values times 10°. 
Table 10.3 * ae 
Simulation variance for the variance estimator (Ke) 
n 
nN 3,000 2,000 1,000 
4,000 153.25 364.14 1,289.79 
3,000 2,449.36 6,854.52 
2,000 33,210.31 


Note: Actual values are the displayed values times 10°. 


Table 10.4 
Ratio of entries in Table 10.1 to corresponding entries in 
Table 10.2 


n 
n 3,000 2,000 1,000 
4,000 0.42 0.26 0.38 
3,000 0.48 0.26 
2,000 0.42 
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11. Discussion 


In a design-based perspective on estimation for two- 
phase sampling designs, one can follow a regression 
estimation approach or a calibration estimation approach. 
We concentrate on the calibration approach to create 
approximately design-unbiased estimators. The extent of the 
information available for the calibration holds the key to the 
efficiency of the estimates. We recognize in this paper that 
there are three different types of auxiliary variables 
associated with two-phase designs. They have different 
information characteristics. From these we define four 
different auxiliary vectors; one for the phase-one calibration 
and the other three for the phase-two calibration. The 
calibration approach is suitable for analyzing the resulting 
estimators in a systematic manner. As the paper shows, this 
approach also leads to a more efficient variance estimator 
than the traditional method for variance estimation in 
two-phase designs. 
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Survey weighted hat matrix and leverages 


Jianzhu Li and Richard Valliant ! 


Abstract 


Regression diagnostics are geared toward identifying individual points or groups of points that have an important influence 
on a fitted model. When fitting a model with survey data, the sources of influence are the response variable Y, the predictor 
variables X, and the survey weights, W. This article discusses the use of the hat matrix and leverages to identify points that 
may be influential in fitting linear models due to large weights or values of predictors. We also contrast findings that an 
analyst will obtain if ordinary least squares is used rather than survey weighted least squares to determine which points are 


influential. 


Key Words: Influence; Linear regression; Survey data; Weighted least squares. 


1. Introduction 


In some conventional linear regression diagnostics, it is 
often useful to measure the influence each data point can 
have in determining the values of parameter estimates and, 
in turn, fitted values. The hat matrix and its diagonal 
elements, referred to as leverages, are popular techniques 
that are used to identify the cases that have outlying values 
for predictor variables, and, therefore, may be influential in 
model fitting if they are also associated with unusual 
residuals. When there is more than one predictor variable 
in the regression, analysts can compute leverages to 
summarize the collective influence of the X values for each 
observation. 

In finite population estimation, a superpopulation 
assumption is usually used to build models. Suppose that 
some model fits reasonably well for the bulk of the 
population. For convenience, we will refer to this as the 
“true” model. However, the goal is usually to find a model 
that has some descriptive or predictive power, bearing in 
mind that no model is really “true”. The influence 
diagnostics should allow analysts to identify points that 
make estimated parameters deviate from that true model. 
Parameter estimates in linear regression using complex 
survey data are often derived from the pseudo maximum 
likelihood approach, outlined by Skinner, Holt and Smith 
(1989, Chapter 3), following ideas of Binder (1983). In this 
paper, we assume that the analyst has decided that an 
estimator involving sample weights is appropriate for his or 
her problem. As shown in later sections, the survey 
weighted hat matrix and leverages are useful for detecting 
potentially influential observations caused by not only 
extreme X values, but also by large sample weights. 

Previous survey literature has discussed the effect of 
outliers on some survey estimates, but does not give much 
attention to diagnostics for linear regression models. Deville 


and Sarndal (1992), and Potter (1990, 1993) discuss some 
possibilities for locating or trimming extreme survey 
weights when the goal is to estimate population totals and 
other simple descriptive statistics. Hulliger (1995) and 
Moreno-Rebollo, Mufioz-Reyes and Mufioz-Pichardo 
(1999) address the effect of outliers on the Horvitz- 
Thompson estimator of a population total. Smith (1987) 
demonstrates diagnostics based on case deletion and a form 
of the influence function. Zaslavsky, Schenker and Belin 
(2001), and Beaumont and Alavi (2004) use M-estimation 
based strategies to downweight the influential clusters or 
units. Chambers (1986), Gwet and Rivest (1992), Welsh 
and Ronchetti (1998), and Duchesne (1999) conduct 
research on outlier robust estimation techniques for totals. 

A perennial question among analysts of survey data is 
whether to use the survey weights or not when fitting 
models. The collections edited by Skinner ef al. (1989) and 
Chambers and Skinner (2003) discuss this issue at length. 
Binder and Roberts (2003, Chapter 3), Chambers, Dorfman 
and Sverchkov (2003, Sections 11.2.3, 11.6), Chambers and 
Skinner (2003, Chapter 1), Korn and Graubard (1999, 
Sections 4.3, 4.4), Pfeffermann (1996), and Smith (1989, 
Chapter 6) describe the arguments pro and con. The details 
can be quite mathematical and abstract but are summarized 
succinctly by Skinner (2003, Section 6.2.3). 

We paraphrase Skinner (2003, Section 6.2.3) here in the 
context of fitting a linear model to predict some response Y 
based on a set of explanatory variables X. If the linear 
model is specified correctly and the sampling depends only 
on the explanatory variables in the model, then unweighted 
regression parameter estimates will be unbiased in a model- 
based sense. In particular, the assumed conditions require 
that the survey weights are unrelated to Y conditional on the 
values of the X predictors. However, if sampling depends 
on factors that may be related to Y, even after conditioning 
on the values of the predictors, the unweighted parameter 
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estimators will be biased both with respect to the true model 
and in the design-based, repeated sampling sense. This 
situation is known as having an informative sample design 
in which the distribution of the sample values of Y is 
different from the population distribution. An example of 
this is given by Chambers, Dorfman and Sverchkov (2003, 
Section 11.2.3). If sample units are selected with 
probabilities proportional to some measure x of their size 
and Y is related to x, the sample distribution of Y will be 
skewed to the nght of its population distribution. The 
situation in this example is similar to the one in our 
empirical study in section 5. 

Using the survey weights guards against the bias that 
may result from not accounting for an informative sample. 
Also, if the model is not correctly specified, the survey- 
weighted regression still estimates a census parameter. That 
is, the weighted estimates are approximately unbiased for 
the best-fitting linear model that would be obtained if the 
entire finite population were in hand. In this paper, we 
assume that an analyst has made the decision to use weights 
in fitting a model, possibly for the reasons above, and 
provide one type of diagnostic for assessing the effects of 
certain data points. 

The hat matrix and leverages we present are the same 
ones that are produced by standard software packages when 
a weighted least squares regression is done. However, the 
literature is missing any discussion of their use and 
interpretation in the context of survey-weighted regression. 
Korn and Graubard (1999) is one of the few references that 
addresses any kind of diagnostics for models fitted from 
survey data. Leverages are among a series of diagnostic 
tools and will be more effective when evaluated with 
residuals. Many diagnostic statistics, such as the famous 
Cook’s distance (Cook 1977) tur out to have both 
leverages and residuals as components. 

The literature gives somewhat ambiguous guidance on 
how to deal with the influential observations once they are 
identified. An obvious, and perhaps naive, solution is to 
remove the outliers and refit the model, which makes sense 
when the outliers result from improperly recorded data. A 
natural extension of this would be to devise an automatic 
approach where certain rules would be used to identify 
influential points, delete them, and refit the model. Our 
presumption in this article is that, after identification of 
influential points and careful consideration of the reasons 
for the influence, an analyst will determine whether the 
points should be excluded from fitting. This is in contrast to 
setting up some procedure that would automatically exclude 
points based on some cutoff values. 

The remainder of the paper is organized as follows. 
Section 2 describes the ordinary least squares hat matrix, 
leverages, and some of their properties. Sections 3 and 4 
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cover the survey-weighted hat matrix and leverages plus a 
decomposition that shows how points can have large 
leverages. The extensions to survey data apply to both 
single- and multi-stage designs. Section 5 gives a numerical 
example using a single-stage sample of mental health 
organizations. The last section summarizes our findings and — 
gives some directions for additional research. 


2. OLS hat matrix 


A working model is one that is being provisionally 
considered by an analyst for the structure that best describes 
a conceptual superpopulation. It may be revised after further 
assessment by adding predictors, dropping predictors, or 
making other changes to the form of the model. Suppose 
that the working linear model is 


Y= Xf te Vig) =.0 | (1) 


where Y = (Y,..., Y¥,)’, X’ = (x, .... x,) with x, = 
Caras 2 b= (bee ad pe ANG Gr et oen (eee es ay 
Assuming the X matrix is of full rank, the ordinary least 
squares (OLS) estimate of B is 


B = (X’X)'X’Y=A'X’Y, (2) 


where A = X’X is a square matrix and invertible. The 
fitted values Y corresponding to the observed values Y 
are 


Y = XB = XA'X’Y = HY, 


where H = XA™'X’ is called the hat matrix. This name 
was first introduced by Tukey (Belsley, Kuh and Welsch 
1980, Chapter 2; Hoaglin and Welsch 1978). The leverage, 
h, =x! A’'x,, is the i" element on the diagonal of the hat 
matrix, which measures the impact of Y, on its own fitted 
value since Y, = Dy hy Y= hy Vp th Qegei By Vj fn 
approaches 1, Y, has a crucial role in determining the value 
of Y,. 


The OLS hat matrix and leverages have many special 
and useful properties: 


(i) H is symmetric, or h, = h,;; 

(i) H is idempotent, or 
H(I — H) = 0; 

(au) HX = X or (I- H)X = 0; 

CN) AO Hoes 

(v) >A, = rank(X) = p, which implies that the 
mean leverage is h = p/n; 


H=H?’, or 


if model (1) has an intercept, the following two properties 
hold: 


(vi) DA, = 1; 
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(vii) 4, = 1/n+ (x, — x)’ A '(x, — X), where X = 
ES efi 

In a reasonably large data set, an individual leverage 
value h,, is usually considered extreme if it is more than 
twice the mean, h = p/n (Belsley et al. 1980, Chapter 2). 
The existence of a gap between most of the cases and a few 
unusual cases in the empirical distribution of the leverages 

also provides evidence of outlying units. 


3. Survey weighted hat matrix 


The initial step in the pseudo maximum likelihood 
approach is to form the set of estimating equations that 
would be appropriate for a model if the entire finite 
population were observed. This set is a type of population 
total which is then estimated using design-based survey 
methods. Suppose that the underlying structural model is a 
fixed-effects linear model: 
¥=x' Bre, 


U 


e, ~ ind N(0, v,0°) (3) 


where €, is independently normally distributed with mean 0 
and variance v,o°, which is known except for the constant 
6°. The pseudo maximum likelihood estimator (PMLE) of 
B is the solution to the set of estimating equations 
X’ WV '(Y — XB) = 0, with V = diag(y, ..., v,) and 
W = diag(w, ..., w,). Survey weights, which in proba- 
bility samples are usually inversely proportional to inclusion 
probabilities, are used in the PMLE to account for an 
informative design in which the sample distribution of the 
Y’s is likely to differ from that of the finite population. 
These equations can be solved explicitly as i = 
(X’ WV! X)'X’ WVY. If we assume V = I, model 
(3) reduces to (1) and the survey-weighted (SW) estimator 
6 will consequently take the form of a weighted least 
squares estimator, B = (X’ WX) 'X’ WY. 

When survey weights are accounted for in the regression, 
the predicted values become Y = HY, where the hat 
matrix includes the survey weights and is defined as 

H = X(X’ WX) 'X’ W = XA7X’ W 
with A = X’ WX. The leverages on the diagonal of the hat 
matrix are h, = x) A’'x,w, In this formulation, it is 
assumed that the analyst does not incorporate a V matrix in 
the regression. However, results can be modified to 
incorporate V simply by using W = WV" rather than 
W. Unlike the unweighted hat matrix, the SW hat matrix is 
no longer symmetric for sampling designs with unequal 
selection probabilities (or, more generally, unequal 
weights). Properties (ii) — (vi) in section 2 still hold (e.g., see 
Valliant, Dorfman and Royall 2000, Chapter 5) provided the 
unweighted hat matrices were replaced by the weighted 
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ones. In addition, the SW hat matrix has extra useful, and 
easily verified, properties as follows: 


a) WH = WXA'X’ W =H’ w; 
b) X’ W(.- H) = X’ W-X’ HW = 0: 


ee If y=! ae 
Cc) wh, = wx, A X,w, = w, hy. 


The definition of the weighted leverages indicates that a 
large leverage may be caused by outlying X values, an 
outlying weight, or both. Note that the formulas for the 
survey-weighted hat matrix and leverages apply regardless 
of whether the sample design uses strata or is single-stage or 
multi-stage. This is in contrast to diagnostics, like Cook’s D, 
that require estimated standard errors or covariance matrices 
that should be specialized to fit the sample design. 


4. Decomposition of leverages 


Leverages can be decomposed into components that 
separate the effect of the weight and the X values for a 
unit. Suppose the working model is (1) and that the model 
contains an intercept, so that 


x) x} 
DG CRY 9 FR, G0 er ra A, Sa i 
ex? x 


where x, = (Xj, ---> X;,p-1) are 1x (p — 1) vectors, 1 isa 
nx1 vector with all the elements equal to 1, and X, isa 
n x (p — 1) matrix. The A matrix is computed as 


Vv 1’ Wi 1’ WX, Nt, 

A= W(1 X,) = =|. , 
xt X; W1 X) WX,) (tA, 

where t y is a (p-—1)x1 vector with elements t y= 


Dies W, x, and A, isa (p—1)x(p-J) matrix. Using the 
inverse of a partitioned matrix, 


] ] TT NN ] 1 ALE ek 
— +--t St = — —t S 
eel cna eae Ty nue 
~—S"'t, Ss” 
a) Saale oa 
“A Ww Ww W 
a Wh 
-$'X, Ss} 
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where xX, =t,/N is a (p—1)x1 vector, and S = 
Ai t,t, /N isa (p-1)x(p-—1) matrix. Simplifying 
the hat matrix using the above inverse matrix, we obtain 


H = XA7'X’ W 


- eau + (X, -1x7) S"(-x,17 + xT} w 


I =T 
x, iF Xy 


Then, using the fact that N = nw with w= >",w,/n, the 
leverage of i observation, or the i" diagonal element of 
the weighted hat matrix H, is 
h, = + “t+ N(x, -%,)?S"(x, -¥,)]. 
nw 

The quadratic form, (x, — X,)’ S”' (x, —X,,), defines an 
ellipsoid centered at X, (e.g., see Weisberg 2005, Chapter 
8), and N(x, —xX,)’S'(x,—X,) is the Mahalanobis 
distance from x, to X,. Consequently, a leverage can be 
large if (1) w, 1s large, especially relative to the average 
weight w; or (2) x, is far from the weighted average, X,,, 
of the X, in the metric determined by the matrix S. 

For example, in a simple linear model with only one 
auxiliary variable, y, = a+ Px, +6, ©, ~ (0, o’), the 
leverage of the i” observation is 


nw Dia W, (x; — Xp) 
where X, = >, w,x,/N. 
If the error terms in the model have a general variance 


structure ¢ ~ (0, V) and V is known, the hat matrix is 
then defined as H = XA!) X7 WV" with 


WV '1 1’ WV'X, | 
A 


X/ V'WI1 X] WV'X, 
Dw! y, wx; /y, 
> wx, /v, > wxx, /y, | 


A formula for Aq like the one above applies with t,, = 
Y.,x,/v, Ny = ,w,/v, and S,=X?wv'x,- 
t,, t1,/N,. If a general V is used, ty and N, no 
longer are design-based estimates of T, and N but are 
estimates “of Th, xv. and? iV, = 5,471. The 


leverage of the i observation under this general model is 
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W; 


i= 
v. Ny 


W 


[1+ N, (x; —Xw)' S/' (&; —Xwv)I- 


5. Numerical example 


As noted in section 1, arguments can be advanced to 
justify ignoring sample design features, generally, and 
weights, in particular, when fitting models. Roughly 
speaking, when a model conditions on all the design 
variables determining the sampling scheme and the model is 
correct for both the population and the sample, OLS 
regression can be used. Analysts may object to including 
design variables in a model because some are not 
scientifically interesting as predictors. In addition, 
conditioning on all design variables may not be possible, 
especially when the “sampling scheme” includes 
uncontrolled nonresponse that itself may be related to the 
response variable. As noted in section 1, SW provides a 
modicum of protection against having a misspecified model 
when the distribution of the sample Y’s is different from 
that of the population due to the type of sample design used. 
Nevertheless, some analysts will contend that the sample 
design and survey weights can be ignored in specific 
applications and that OLS is appropriate. Thus, it is 
interesting to see how different the OLS diagnostics are 
from SW diagnostics in a real application. However, given a 
course of action, an analyst should use diagnostics 
consistent with the method of fitting. If OLS is used, the 
standard OLS diagnostics should be examined; if SW 
regression is used, SW diagnostics are appropriate. It may 
well be that different points are influential depending on 
whether one uses OLS or SW regression. 

In this section we examine the hat matrix and leverages 
in a regression example using the 1998 Survey of Mental 
Health Organizations (SMHO) conducted in the U.S., which 
collected data on specialty mental health care organizations 
and general hospital mental health care services. The sample 
for this survey was based on a stratified single-stage design 
with probability proportional to size (PPS) sampling 
(Manderscheid and Henderson 2002; Choudhry 2000). The 
measure of size (MOS) used in sampling was the number of 
“episodes”, defined as the number of patients/clients of an 
organization at the beginning of 1998 plus the number of 
new patients/clients added during calendar year 1998. Many 
of the analysis variables in the survey are related to the 
MOS, and their unweighted sample distributions will be 
different from the population distributions since the sample 
tends to have larger size units. Thus, this design is 
potentially informative as defined in Chambers and 
Skinner (2003). 
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The varying sizes of the mental health care organizations 
resulted in the values of collected variables in the sample 
having wide ranges, which may cause some observations to 
have relatively large influence on the parameter estimates of 
a linear regression. The model of interest in this study is to 
regress the total expenditure of a health organization, in 
1,000’s of dollars, on the number of beds set up and staffed 
for use and the number of additions of patients or clients 
during the reporting year. The SW _ estimator, i & 
(X’WX) 'X’WY, was used. Mimicking the procedure 
employed by most analysts, we did not incorporate a model 
variance matrix V in the estimate of the regression 
parameter. A total of 875 observations was used in the 
regression, each of which had non-missing values on the 
independent and dependent variables. 

Table 1 gives a summary of the quantile values of the 
variables involved in the regression, including the survey 
weights. The total expenditure has a maximum of 
519,863.3, which is almost 30,000 times the minimum, 
16.6. Although not as extreme as the total expenditure, the 
number of beds and the number of additions also have 
significant differences between their maxima and minima. 
Because the sample was selected using a PPS design, the 
sample weights were associated with the sizes of the mental 
health organizations, with a range from | to 158.86. The 
weights we use in analysis include a nonresponse 
adjustment which was done separately by design stratum. In 
some cases, units that were selected with certainty in the 
initial sample did not respond and some of the responding 
certainties had their weights adjusted to be larger than 1. A 
total of 157 organizations had a weight of 1 after the 
nonresponse adjustment. 


Table 1 
Quantiles of variables in SMHO regression 

Quantiles 
Variables 0% 25% 50% 15% 100% 
Expenditure (1,000’s) 16.6 2,932.5 6,240.5 11,842.6 519,863.3 
# of Beds 0 6.5 36 93 2,405 
# of Additions O 558.5 1,410 2,406 79,808 
Weights 1 1.42 2.48 Vets) 158.86 


In the regressions that follow, we have included the units 
with weights of | in standard error estimation rather than 
excluding them, as would be the approach for handling 
certainties in purely design-based estimation. Including the 
certainties is consistent with the idea that a superpopulation 
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model is being estimated and that slope coefficients would 
still have a variance even if a census were done. A sketch of 
the mathematical justification for doing this is model- 
dependent (not design-based) and is given in the Appendix. 

Figure 1 shows scatterplots of expenditures versus beds 
and additions for the sample of 875 facilities (omitting one 
extremely large facility described below). In the first row, 
points are highlighted whose OLS leverage is greater than 
2p/n =0.007. The second row shows bubbleplots with the 
relative size of the bubbles proportional to the weight of 
each case. High SW leverage points are highlighted using 
the same cutoff of 0.007. The distributions of the predictors 
are quite skewed as noted in Table 1. There is also one very 
large facility that is not shown in Figure 1 because it distorts 
the scale of the plot. That facility (denoted as observation 
818 here) has (expenditures in 1,000’s; beds; additions) = 
($519,863.3; 2,405; 79,808) and has a survey weight of 
2.22. (Observation 818 was one of the cases noted earlier 
that was a certainty in the initial sample but received a 
nonresponse adjustment, and, thus, had a final weight larger 
than 1.) Because its data values are far out of line with those 
of the other organizations, this point has the potential to 
affect estimates. 

Table 2 reports the twenty observations with the largest 
SW leverages. The values of the leverages range from 0.022 
to 0.389, substantially greater than the level of the rough 
rule of thumb 0.007. This table also shows, for these twenty 
cases, the OLS unweighted leverages, the ratio of individual 
sample weight to average sample weight and the relative 
absolute distance between individual X values and their 
weighted means. We note that unit 818 has the highest 
weighted and unweighted leverages, mainly resulting from 
its extremely large number of beds and number of additions. 
Since this case has a less-than-average sample weight, the 
OLS leverage is even larger than the weighted one. There 
are other similar cases such as units 271, 179, 820, 157, 163, 
156, and 154, which are associated with either extreme 
number of beds, or extreme number of additions, or both — 
but have small weights. Another type of outlier results from 
extreme sample weights, even if the values of their auxiliary 
variables are not very distinct from others. Units 672, 613, 
711, 801, and 611 all have sample weights more than 15 
times the average weight. Their weighted leverages are 
identified as large, whereas the unweighted leverages are 
not. There is also a noticeable gap between the weighted 
leverages for case 331 (h,=0.075) and for case 271 
(h,, = 0.046). 
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Figure 1 Scatterplots of expenditures versus beds and additions. High leverage points 
based on OLS (SW) are highlighted in top (bottom) row 


Table 2 
Observations with 20 largest survey weighted leverages 
Weights Beds Additions 
Obs ID OLS h;; Weighted h,; w; |W Pr ex alee, Ixy, eee Lie 
818 0.513 0.389 0.3 49.3 64.7 
189 0.037 0.245 3.4 ii 0.3 
346 0.035 0.157 2.2 0.6 16.1 
366 0.017 0.105 3.0 0.7 Lda 
331 0.024 0.075 1.5 0.1 13.4 
271 0.068 0.046 0.4 23. 0.0 
830 0.004 0.045 5.8 5.4 0.1 
628 0.056 0.045 0.4 1.0 20.3 
179 0.089 0.038 0.2 27.4 0.5 
672 0.002 0.034 24.2 1.0 0.8 
820 0.048 0.034 0.3 0.8 19.6 
207 0.012 0.030 1.3 75 0.3 
og 0.069 0.030 0.2 23.8 0.5 
163 0.017 0.027 0.8 11.4 0.8 
613 0.002 0.026 18.5 1.0 0.7 
711 0.002 0.024 16.8 1.0 0.9 
801 0.002 0.024 ite 0.6 0.9 
156 0.055 0.023 0.2 20.9 0.9 
611 0.002 0.023 15.9 1.0 0.8 
154 0.051 0.022 0.2 20.5 0.1 
ey a. x =47.83 % = 121413 


Note: observation ID is the line number of an observation in the sample. 
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Sizes of the sample weights can make analysts reach 
different conclusions when they use weighted or un- 
weighted leverages to identify potentially influential obser- 
vations. Figure 2 shows a scatterplot of weighted leverages 
versus unweighted ones. The two reference lines were 
drawn at values of 0.007. Observation 818 is omitted since 
it would again distort the scale of the graph. Clearly, the 
high leverage points identified by the SW method only, 
located in area A, have significantly larger weights than the 
points in area B, which are identified by the OLS method 


SW leverages 


0.10 4025 


0.05 


0.00 


T a F 


0.00 0.02 0.04 0.06 0.08 
OLS leverages 


Figure 2 Plot of survey weighted leverages versus OLS 
unweighted leverages 


Given that some potentially influential cases have been 
identified, the next step is to see what effect they have on 
parameter estimates. Table 3 shows the OLS and SW 
parameter estimates using all cases. Table 4 lists the OLS 
and SW estimates (i) omitting high leverage cases and (ii) 
omitting observation 818. High leverage points are those 
with h,, > 0.007. However, note that different sets of points 
are high leverage in OLS and SW regressions. The standard 
errors are estimated via the usual OLS formula and the 
sandwich estimator (Binder 1983) for the SW estimates. 

Comparing Tables 3 and 4, we see that the OLS 
estimates change substantially after high leverage points are 
deleted (section (i) of Table 4). The OLS intercept, which is 
significant in both tables, jumps from negative to positive. 
The OLS slope for beds drops by about 26% (94.16 to 
69.27) when the high leverage points are dropped. The 
decrease is about 59% for the slope for additions. The SW 
estimates for beds and additions are also sensitive to the 
high leverage points with the slopes decreasing by 7% and 
46% respectively. In all cases, the slopes are significant so 
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that the qualitative conclusion that expenditures is related to 
beds and additions holds with or without the high leverage 
points. However, predicted values will be quite different 
before and after omitting these points. 

The standard errors (SE’s) also decrease substantially 
when the high leverage points are omitted. For example, the 
SW standard error for beds drops from 13.14 to 6.75 (a 49% 
reduction); the SE for additions drops from 0.76 to 0.21 (a 
72% reduction). This is due to some points with extreme 
weights being removed in the SW regression. In contrast, 
the SE’s for the OLS estimates actually increase when the 
OLS high leverage points are omitted because the sample 
variance of the x’s decreases. This is another illustration of 
the considerable differences that can occur when applying 
the same type of diagnostic to OLS and SW regressions. 


Table 3 
OLS and SW parameter estimates of SMHO regression using all 
875 sample cases 


Independent OLS Estimation SW Estimation 
Variables Coefficient SE t Coefficient SE t 
Intercept =1 201-73" 52619" =208" “5S 1408" “1 137:7F 0A4 
# of Beds 94.16 3.03 31.08 81.23 13.14 6.18 
# of Additions 2.31 0.13 18.50 1.84 0.76 2.43 
Table 4 

OLS and SW parameter estimates after from SMHO regression 
Independent OLS Estimation SW Estimation 
Variables Coefficient SE t Coefficient SE t 


(i) Deleting observations with leverages greater than 0.007 


Intercept 2,987.55 490.54 6.09 1,993.86 353.71 5.64 
# of Beds OLY 4.35 15.94 WS-82 Coytion alii 28) 
# of Additions 0.95 0.20 4.71 1.00 O21 5 493 
(ii) Deleting observation 818 

Intercept 1979 SIGS 7-98) 32O8e 22817. A600 355 4.96 
# of Beds 81.80 DS WY OSS 68.69 8.04 8.54 
# of Additions al 0.14 8.41 0.79 29) 2 


Because point 818 is so obviously extreme, we also fitted 
the regression after dropping only that observation. The 
results are shown in section (i1) of Table 4. Omitting that 
single point causes noticeable changes in both OLS and SW 
parameter estimates. This also illustrates that a single point 
can affect the standard errors for estimated slopes in a 
survey-weighted regression, as is also the case in OLS. 
Observation 818 has a large residual (see Figure 3); omitting 
it results in the SE for Beds dropping from 13.14 in Table 3 
to 8.04 in Table 4. Note that if unit 818 had a large weight, 
then its residual would likely be smaller since it would have 
more affect on the fit. If so, the SE could actually be smaller 
when unit 818 is included. 
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Another point to be gleaned from Tables 3 and 4 is that 
the OLS and SW estimates are much closer to each other 
after the high leverage points are dropped than they are 
before. As shown in Table 5, the OLS estimates are 16 and 
26% larger than the SW estimates with all points but are 9 
and 5% less than SW after dropping points. 


Table 5 

Ratios of OLS and SW parameter estimates before and after 
deleting observations with leverages greater than 0.007 from 
SMHO regression 


Ratio of OLS to SW estimates 
With all points Dropping high leverage 
points 
Beds 1.16 0.91 
Additions 1.26 0.95 
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Figure 3 Plot of fitted values versus Y values. Reference line 
is drawn at Y=¥Y. The upper panel includes all 
points. The lower panel omits the extreme 
observation 818. High leverage points based on SW 
are solid, dark circles in each panel 
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Leverages are usually combined with residuals to deter- 
mine which points are influential in fitting the regression 
model because residuals can be used to detect discrepant Y 
values. A scatterplot of fitted values from the SW regression 
versus the Y values is shown in Figure 3. The high leverage 
points are labeled as dark solid circles. The vertical 
distances from the points to the 45 degree line imply the 
sizes of the residuals. The upper panel includes all 875 
sample points; the lower panel omits observation 818 to 
provide better resolution for the remaining points. Note that 
some observations have high leverages and small residuals, 
while others have low leverages and large residuals. The 
influence of these points on the regression can be further 
investigated using various tools that we will not cover here. 
For example, Cook’s distance, implicitly involving the 
leverage and residual, is designed to measure the effect of 
deleting a single observation on the overall parameter 
estimates. The adaptation of some basic OLS diagnostic 
statistics to survey data, such as DFBETAS and DFFITS, 
has been discussed under a single stage sampling design in 
Li and Valliant (2006). 


6. Conclusion 


Leverages and residuals are essential components of 
diagnostic statistics intended to identify substantial influ- 
ence of a single observation or a group of observations on a 
fitted linear model. Survey data sets can contain influential 
observations whether one argues that the sample design is 
ignorable and ordinary least squares can be used, or that the 
design must be accounted for and survey weights used. The 
points that are influential in the two cases are not necessarily 
the same, as illustrated here. 

Once high leverage points are identified, an important 
question is how to deal with them for inference. Two 
options are to down-weight them or drop them from model- 
fitting entirely. Down-weighting seems unsatisfactory in 
general since a point can have a high leverage not because 
of a large weight but rather due to having one or more 
unusual X°s. Down-weighting may be sensible from a 
model-based point-of-view, assuming the model itself is 
correctly specified. However, the design-based idea of 
estimating a census parameter may then be lost. If a point 
has a large leverage because of extreme X’s, then it may not 
follow the model at all and should be dropped. 

However, using a mechanical procedure that auto- 
matically drops many influential observations with high 
leverages can lead to standard error estimates that are too 
small, resulting in confidence intervals that cover at less 
than the nominal rates and in inflated Type I error rates in 
hypothesis tests (Li 2007). This phenomenon is similar to 
well-known problems in stepwise regression (Hurvich and 
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Tsai 1990, Zhang 1992). Thus, a useful research topic 
appears to be developing inferential procedures for 
constructing confidence intervals and conducting hypothesis 
tests that account for the effects of dropping or down- 
weighting points. 

For complex survey data, the hat matrix involves no 
design features except for sample weights and can be used 
to identify cases that have atypical weights or predictor 
values. Other diagnostic statistics, like Cook’s D, do contain 
variance estimates that need to account for complex sample 
design features such as stratification and clustering. The 
adaptation and extension of additional diagnostic ap- 
proaches for survey analysis will be explored in the future. 
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Appendix 


Inclusion of certainties in standard 
error estimation 


In the empirical study in section 5, we included certainty 
units in the standard error calculations. The justification for 
doing this is sketched here. Under the general model (3), 
the model variance of B = (X’ WX) |X’ WY, the esti- 
mator used in the empirical study, is var,, (B)=A 1X7 
WVWXA 'o° where A = X’ WX and V= diag(v,),... 
The sandwich variance estimator used in the study reported 
in section 5 is defined as 


n 


v(B) = Av : Danka si ZZ ZA Ny kA) 


n 


where z,=w,e,x, with e, =Y, = xp and Z= 
Dies W, e, X,/n. This estimator is design consistent (see 
Binder 1983) in single-stage sampling if units are sampled 
with replacement with probabilities equal to w;', and there 
are no certainty units. If the sample contains certainties, the 
formula for v(B) would be modified to estimate the design- 
based variance: certainties would be excluded from the 
sums in (4) and Z, and n would be changed to n,., the 
number of non-certainties. In the extreme case of a census, 


the design-based variance estimator would reduce to zero. 
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The estimator in (4) is approximately model-unbiased 
under (3) regardless of whether the sample contains 
certainties or not. The middle matrix in (4) can be expanded 
as Lies(Z; — Z)(Z; - Z)" =e, Biz onze Assuming 
that e, ~ Y, — x/B, the model expectation under (3) of the 
first term is Ey, (DjesZ,Z,) = X’ WVWXo° while 
E,, (nzz') = n'X'’WVWXo’. Substituting these expec- 
tations gives E,, [v(B)] = vary, (B), which holds even 
when some units are certainties. This also shows that v(B) 
is robust in the sense of properly reflecting the contribution 
of heteroscedastic variances in (3) to the model-variance of 
i even though V may be unknown and not accounted for in 
the estimation of B. 
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A practical bootstrap method for testing hypotheses from survey data 


Jean-Francois Beaumont and Cynthia Bocci ' 


Abstract 


The bootstrap technique is becoming more and more popular in sample surveys conducted by national statistical agencies. 
In most of its implementations, several sets of bootstrap weights accompany the survey microdata file given to analysts. So 
far, the use of the technique in practice seems to have been mostly limited to variance estimation problems. In this paper, we 
propose a bootstrap methodology for testing hypotheses about a vector of unknown model parameters when the sample has 
been drawn from a finite population. The probability sampling design used to select the sample may be informative or not. 
Our method uses model-based test statistics that incorporate the survey weights. Such statistics are usually easily obtained 
using classical software packages. We approximate the distribution under the null hypothesis of these weighted model-based 
statistics by using bootstrap weights. An advantage of our bootstrap method over existing methods of hypothesis testing 
with survey data is that, once sets of bootstrap weights are provided to analysts, it is very easy to apply even when no 
specialized software dealing with complex surveys is available. Also, our simulation results suggest that, overall, it performs 
similarly to the Rao-Scott procedure and better than the Wald and Bonferroni procedures when testing hypotheses about a 


vector of linear regression model parameters. 


Key Words: Bootstrap weights; Analysis of survey data; Hypothesis testing; Informative sampling; Linear regression; 


Model parameters. 


1. Introduction 


The bootstrap technique is becoming more and more 
popular in sample surveys conducted by national statistical 
agencies. The main reasons seem to be that it can easily deal 
with several situations that would be difficult to handle 
otherwise (e.g., nonresponse weight adjustment, calibration, 
non-smooth statistics, efc.) and that it is convenient for 
analysts. In most of its implementations, several sets of 
bootstrap weights accompany the survey microdata file 
given to analysts; no other design information is provided. 
These weights are usually obtained by assuming that the 
first-stage sampling fractions are small enough that a 
without-replacement sampling design can be accurately 
approximated by a with-replacement sampling design. The 
reader is referred to Rao, Wu and Yue (1992) for a succinct 
but clear description of a method to construct bootstrap 
weights under this assumption when a stratified multistage 
sampling design has been used. 

So far, the use of the technique in practice seems to have 
been mostly limited to variance estimation problems (e.g., 
Langlet, Faucher and Lesage 2003; Yeo, Mantel, and Liu 
1999; and Hughes and Brodsky 1994). On the research side, 
efforts have been mainly oriented towards finding an 
appropriate bootstrap methodology for variance estimation 
when the sample is drawn without replacement from a finite 
population (see Sitter 1992; or Shao and Tu 1995, Chapter 
6, for a review of methods). Some authors have also studied 
the problem of determining bootstrap confidence intervals 


for a finite population parameter (e.g., Rao and Wu 1988; 
Kovar, Rao and Wu 1988; Sitter 1992; and Rao et al. 1992). 
To our knowledge, there does not seem to be any literature 
on hypothesis testing using the bootstrap technique in 
survey sampling although this problem has been studied in 
the context of classical statistics. The reader is referred to 
Hall and Wilson (1991) for a discussion on bootstrap tests 
of hypotheses and to Efron and Tibshirani (1993) for an 
excellent account of the bootstrap technique in classical 
statistics. It is worth noting the work of Graubard, Korn and 
Midthune (1997) who applied the classical parametric 
bootstrap method to survey data in order to test the fit of a 
logistic regression model. Their procedure is valid when 
sampling is not informative. 

The problem of hypothesis testing from complex survey 
data has been well studied in the last 30 years (e.g., Rao and 
Scott 1981; Fay 1985; Thomas and Rao 1987; Kom and 
Graubard 1990; Korn and Graubard 1991; Graubard and 
Korn 1993; Thomas, Singh and Roberts 1996; and Rao and 
Thomas 2003). However, except perhaps for estimating 
unknown variances/covariances involved in these methods, 
the bootstrap technique has apparently not yet been 
considered for testing hypotheses. The goal of this paper is 
thus to propose a bootstrap methodology for testing 
hypotheses about a vector of unknown model parameters 
when the sample has been drawn from a finite population. 
The probability sampling design used to select the sample 
may be informative or not. Informally speaking, sampling is 
informative when the model that holds for the selected 
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sample is different from the model that holds for the whole 
population; otherwise sampling is not informative. 

Our method uses model-based test statistics that 
incorporate the survey weights. Such statistics are usually 
easily obtained using classical software packages. We 
approximate the distribution under the null hypothesis of 
these weighted model-based statistics by using bootstrap 
weights. An advantage of our bootstrap method over 
existing methods of hypothesis testing with survey data is 
that, once sets of bootstrap weights are provided to analysts, 
it is very easy to apply even when no specialized software 
dealing with complex surveys is available. 

We introduce notation and the problem in section 2. In 
section 3, we describe and justify our proposed bootstrap 
methodology for testing hypotheses with survey data. A 
linear regression example is given in section 4 to illustrate 
the theory. We briefly describe the alternative Rao-Scott 
(Rao and Scott 1981), Wald and Bonferroni procedures in 
section 5 when testing hypotheses about a vector of linear 
regression model parameters. They are evaluated in section 
6 and compared to our proposed bootstrap procedure 
through a simulation study. Finally, we conclude in the last 
section with a short summary and discussion. 


2. Preliminaries 


We assume that a finite population U of size N has 
been generated according to a model, specified by the 
analyst, that describes the conditional distribution 
F(yy|Xy; B, 9). The N-vector y, contains the 
population values of a dependent variable y, X,, is an N- 
row matrix that contains the population values of a vector of 
independent variables x, B is an r-vector of unknown 
model parameters and @ is a potential vector of additional 
unknown model parameters. We are interested in testing 
hypotheses about B but not 6. We also assume that, if the 
entire population U could be observed, a test statistic 
t(U; c) would be used to test the multiple linear 
hypothesis H, : HB = ¢ against the alternative hypothesis 
H,: HB # c. The Qxr matrix H is used to define the 
hypothesis to be tested and ¢ is a Q-vector of constants 
specified by the analyst. Ideally, ¢(U; ¢) is asymptotically 
pivotal; i.e., it has an asymptotic distribution that does not 
depend on any unknown parameter. We consider statistics 
that have the following quadratic form: 


t(U; c) = (HB, — )' {AWU)} (HB, -¢), (2.1) 


where f,, is a consistent estimator of B under the model 
and A(U) is some scaling matrix. Typically, A(U) is 
symmetric and positive definite. 

As an illustrative example, let us assume that y,, for all 
population units k € U, are independently and identically 
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distributed random variables with mean B and variance 8 
and that we are interested in testing the null hypothesis 
H,: B =c. In this example, Q=1,r, =1, H=1 and 
X, =1y, where 1, is a population vector of one’s. A 
common test statistic for this problem is 

A 2 

brenis pen asian 

8, /N 
where 6, =Yuev ¥,/N and 6, =Yuev (% —By)/(N -)). 
The statistic (2.2) has the same form as (2.1) if we let 
AO yrs 6,,/N. This statistic is usually assumed to follow 
the distribution y; or Fj, y_; under the null hypothesis. 

As is typically the case, a random sample s of size n is 
selected from the finite population U according to a given 
probability sampling design p(s). Since the dependent 
variable y and, possibly, the independent variables x are not 
observed for nonsample units, we may want to use the 
statistic t(s;c) instead of t(U;c). In the above example, 
this would lead to t(s;c) = n(p, - c)/6,, where B, = 
Dies ¥,/n and 6. = peli Bn —1). However, if 
sampling is informative with respect to the model, it may be 
more appropniate and is undoubtedly more common to use a 
weighted test statistic of the form 


7 (s, w,; ©) = (HB,, — ©)'{A(s, w,)} '(HB,, — ©). (2.3) 


The n-vector w, contains the survey weight of sample 
unit k in its k" element, denoted by w,, B,, is a weighted 
estimator for B and A (s, W,) is a weighted analogue to 
A(s) in that each sample unit k is weighted by its survey 
weight w, whereas there is no weighting with A(s). We 
thus have A (s, 1.) = A(s), where 1, is a sample vector 
of one’s. As a result, the statistic f(s, w,; ¢) is also a 
weighted analogue to ¢(s; ¢) and we have 7(s, 1,; c) = 
t(s; c). If the statistic ¢(s; c) can be computed using 
some classical software package, not necessarily developed 
to handle survey data, the statistic 7 (s, w,; ¢) can also be 
computed using the same software package provided that it 
can allow each observation to be weighted by its survey 
weight. 

Typically, the survey weight w,, for a unit k es, is 
equal to the inverse of its selection probability, which may 
then be calibrated to account for known extemal 
information (e.g., Deville and Samdal 1992). We assume 
that the sampling design and the survey weights are 
constructed so that the following two assumptions hold: 


Assumption 1: Jn (6. —p) —“—> N(0, 3), where 
—“?—» denotes convergence in distribution under the 
model and the sampling design, and ZX is the asymptotic 
variance-covariance matrix of ./n B,,, under the model and 
the sampling design. The notation “m” stands for the 
model while the notation “p” stands for the probability 
sampling design. 


(2.2) 
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Assumption 2: nA(s, W,) is symmetric, positive definite 
and mp-consistent for some fixed symmetric positive 
definite scaling matrix A. 


Note that assumption 2 does not require A (s, w,) to be 
p-consistent for A(U). Indeed, NA(U) will be typically 
m-consistent for A. Other choices could replace the 
weighted scaling matrix A (s, w,) in (2.3). For instance, it 
could be replaced by an estimator of the design variance of 
HB,,. under simple random sampling (e.g., Rao and Scott 
1981). An alternative choice is the common Wald statistic. 
It is obtained by replacing A(s, WwW) “an (2:3)! by 
vs | (HB,,,.), which is an mp-consistent estimator of 
V,,» (HB,,,); the variance of HB,,, evaluated with respect 
to the model and the sampling design. As pointed out in the 
paragraph below (2.3), an advantage of using a scaling 
matrix A(s, w,) such that A(s, 1,) = A(s) is that the 
resulting test statistic 7(s, w,; ¢) can then be directly 
computed using classical software packages provided that 
they allow each observation to be weighted by its survey 
weight. It is thus more convenient for the users of survey 
data. 

Continuing the above example, we may define our 
weighted test statistic as 


(B,,. — ©) 


f (s, ) Ue 2) dace sop aap i. ReaE ean 
NSN) tai Oya ) 


(2.4) 


where N = >,., %, B.,. = > pes Wi Viel Dukes, Wy. and 6... = 
Dies %(, — By.) /(N -1). In (2.4), the underlying 
weighted scaling matrix is A(s, w,) = {(N —l)/(n—1)} 
(6,,. /N), which does not depend on the way the weights are 
scaled. If they are rescaled so that ¥,.,w, =, which is 
typically done by analysts, then the factor (NW — 1)/(n — 1) 
vanishes. The role of this factor, along with other regularity 
conditions, is to satisfy assumption 2. If the SAS® System is 
chosen, the test statistic (2.4) is obtained by using the 
WEIGHT statement in standard procedures. When the null 
hypothesis is true, it is well known that (2.4) unfortunately 
does not follow the distribution yj or Fy ,.; under the 
model and the sampling design. 

To obtain a valid test procedure, we need to approximate 
the distribution of 7 (s, w,; ¢) under the null hypothesis. 
This can be achieved by using the following result: 


Result 1: ¢ (s,w,; HB) P+? | A,Q,, where 1,, 
for gl" O, **arelithe @ eigenvalues) of})--Ac= 
(A’') (HZH’) and Q., are independent chi-square random 
variables with one degree of freedom. 


The proof of result 1 uses assumptions | and 2 and is 
given in the appendix. When the null hypothesis is true (Z.e., 
HB = c), we thus have 


ri 


Q 
f(s, w,; €) —P-> A, Q,. 
q=) 


(2.5) 


Rao and Scott (1981) used a similar result to construct 
their test procedures. They approximated a distribution like 
(2.5) by a scaled chi-square distribution that matches the 
estimated first two moments of the right-hand side of (2.5). 
Instead, we approximate the distribution of 7 (s, w,; c) 
under the null hypothesis by using bootstrap weights. This is 
described in the next section. 

Before giving details of our test procedure, it is useful to 
note that 7 (s, w.; ¢) in (2.3) can be written as 


aye 


f(s, w,; ¢) = f(s, w,; HB) 
+ 2(HB,,,— HB)’ {A(s, w,)} "(HB - c) 


+ (HB - c)' {A(s,w,)} | (HB-¢). (2.6) 


Under the null hypothesis, the last two terms on the nght- 
hand side of (2.6) vanish and we have f(s, w,; c) = 
i (s, w,; HB). When the null hypothesis is false, the third 
term on the nght-hand side of (2.6) dominates the others as 
the sample size increases since the first, second and third 
terms are O,(1), O, (Jn) and O,(n) respectively, 
provided that assumptions 1 and 2 hold. Also, since 
A(s, Ww.) is positive definite, the third term is always 
positive. Therefore, a large positive observed value of 
f(s, W,; ©) compared to a large percentile of the 
distribution of 7 (s, w,; HB) is an indication that the null 
hypothesis may be wrong. 


3. The proposed bootstrap method 


Let w, denote a random bootstrap weight for unit k, 
obtained using some bootstrap procedure such as that of 
Rao etal. (1992), and let w. be the v-vector that contains 
the random bootstrap weight w, in its k" element. The 
bootstrap estimator De is obtained similarly to 5. by 
replacing the survey weight w, by its bootstrap version Ww, 
for each sample unit. We also denote by w.’, for 
b =1,..., B, the B n-vectors containing the bootstrap 
weights w,” in their k" element. These B vectors are 
drawn independently and have the same distribution as w, ; 
this distribution is called the bootstrap distribution and is 
denoted by the symbol ‘*’. The 5" bootstrap estimator 
pe is defined in an obvious manner. 

Before describing our bootstrap test procedure, we first 
introduce three additional assumptions related to the 
construction of the bootstrap weights: 


Assumption 3: Vn (B.... ~ B.,.. ) ——_» N(0,2), where 
———-» denotes convergence in bootstrap distribution and 
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% is the asymptotic bootstrap variance-covariance matrix 
of Jn B .. 

Assumption 4: nA(s, w_) is *-consistent for nA (s, w,). 
Assumption 5: & is mp-consistent for =. 


Assumptions 3 and 4 are bootstrap analogues to 
assumptions 1 and 2 and should be satisfied with most 
bootstrap methods (e.g., those described in the review paper 
by Sitter 1992) and models (e.g., linear regression model, 
logistic regression model, efc.). The reader is referred to 
Shao and Tu (1995, Chapter 6; in particular section 6.4.4) 
for greater detail. 

A comment is in order about assumption 5. This 
assumption is equivalent to requiring that the bootstrap 
variance V, (B,-.) be mp-consistent for 


Vi (Buoi By vethy, jet VoEM(Be) Rae Gp 


This means that the bootstrap distribution must reflect the 
variability due to both the model and the sampling design. 
Unfortunately, standard design-based bootstrap methods 
reflect only the variability due to the sampling design so that 
they only track the first term of the nght-hand side of (3.1). 
Thus, these bootstrap methods do not satisfy assumption 5 
in general. However, when the overall sampling fraction 
n/ N is negligible, the second term of the nght-hand side 
of (3.1) becomes negligible (e.g., see Binder and Roberts 
2003) so that the approximation V,,, (B,,..) * Ea (B,,,.) is 
appropriate and design-based bootstrap methods can be 
used. In many household surveys, the overall sampling 
fraction is actually quite small. Indeed, bootstrap weights 
are often obtained under the assumption that the first-stage 
sampling fractions are small (eg., Rao etal. 1992). 
Developing bootstrap procedures that capture both terms of 
(3.1) is an area for future research. 

Under assumptions 3 and 4, we obtain our second result: 


Result 2: ? (s, w.; HB,,,) ——> ¥?, A,Q4; where i, 
for,’ = En, OP Vares” thes eigenvalues ror AN 
[nA (s, w,)] ((HZH’) and ©, are again independent chi- 
square random variables with one degree of freedom. 


The proof of result 2 is omitted as it is very similar to the 
proof of result 1 given in the appendix. From assumptions 2 
and 5, A is mp-consistent for A. Thus, using results 1 and 
2, the bootstrap distribution of 7 (s, w.; HB...) is asymp- 
totically the same as the mp-distribution of 7 (s, w,; HB), 
which is itself the same as the mp-distribution of 
f(s, W,; ¢) under the null hypothesis; the distribution that 
we want to approximate. This suggests the following 
bootstrap test procedure: 


i) Obtain bootstrap weights, w,”, for kes and 
aD eG 
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ii) Compute 7 (s, w.’; HB.,..), for b= ee) 

iii) Since a large value of 7 (s, w,; ¢) leads to rejecting 
the null hypothesis, compute the observed 
significance level (p-value) as 


# {7 (s, w.’; HB,,,) > @(s, w,; ©)} 
Dern. ma 


The null hypothesis is rejected if this value is lower 
than the significance level a (e.g., 5%). 


Note that the statistic to be bootstrapped is 
f(s, w.; HB,,.) and not f(s, w.’; c). The use of the 
latter would not properly reflect the distribution under the 
null hypothesis and would thus violate the first guideline in 
Hall and Wilson (1991). 

If 7 (s, w,; ¢) is pivotal then the second guideline of 
Hall and Wilson (1991) is also satisfied. The fact that 
t(U; ¢) is asymptotically pivotal certainly helps in 
obtaining a better bootstrap test procedure. However, it does 
not unfortunately guarantee that f(s, w,; ¢) is also 
asymptotically pivotal, particularly when sampling is 
informative. Nevertheless, failure to use a pivotal statistic 
does not invalidate the above test procedure and may not 
reduce its power. But, it may reduce the level accuracy of 
the test. As pointed out by Hall and Wilson (1991), it is 
sometimes appropriate to disregard the second guideline. 
The main advantage of using the simple (possibly non- 
pivotal) statistic f(s, w,; ¢) in (2.3) and the bootstrap 
statistic 7 (s, w.’; HB,,) is that, once bootstrap weights 
have been provided on the microdata file, these statistics are 
easily obtained using classical software packages that ignore 
sampling design features. Moreover, we show in section 5, 
through a simulation study, that our bootstrap test procedure 
performs similarly to the Rao-Scott procedure and better 
than the Wald and Bonferroni procedures. 


4. A linear regression example 


To better illustrate the theory in a practical context, let us 
now assume that, conditional on X,,, the random variables 
y,, for k € U, are independently distributed with mean 
E,, (y, | Xy) = x;,B and variance V, (y, | X,) = 9, where 
X, 1S an r-vector of linearly independent variables for unit 
k. Recall that we are interested in testing the null 
hypothesis H,: HB = ¢ against the alternative hypothesis 
H,: HB # c. Ifthe entire population could be observed, the 
common statistic 


AU hs 


(HB, - 0 (H(S 0 XXL) H’) (HB, - ¢) 


4.1 
06, = 
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could be used, where 


B, ne (Su) X,X; eo Dees XV 


and 


g, = Das = xB 


N-r 

The statistic t(U; ¢) in (4.1) follows the distribution 
Fo.n-r under the null hypothesis. It reduces to (2.2) when 
QO-r=H=x, =1 in (4.1). 

A weighted sample version of (4.1), which can be written 
in the form of (2.3), is 


i(s, W,; ©) = 


(HB,,, - '(H esas een HW’) (HB,,, - ©) 


aes (4.2) 
COIN —T)iln =r) 
where 
A -] 
B..s a Daw W,X,X, ) aS WeXe Ns (4.3) 
and 
6 = es MOK —x,B,,,)” : (4.4) 
v ae 


For instance, the statistic ¢(s, w,; ¢) in (4.2) could be 
obtained by using the WEIGHT statement in the procedure 
REG of SAS as long as w, > 0, for k € s. Note that it 
satisfies assumption 2 and does not depend on the way the 
weights are scaled. Again, if the weights are rescaled so that 
Dies =n, the factor (N — r)/(n — r) in (4.2) vanishes. 
The test statistic (4.2) reduces to (2.4) when Q=r= 
Hx ae = 1 in (4.2), (4.3) and (4.4). The - bootstrap statistic 
f(s w. HB, ;) as well as B. -, and Oy are obtained 
eae to ¢(s, w,; ©), B,,. and 6... in "4. 2), (4.3) and 
(4.4) respectively, except that w, is replaéed by w,” ande 
is replaced by HB... 


Remark 1: Note that w,” is likely to be 0 for some units 
k € s (see, for example, Rao et al. 1992). In some software 
packages such as SAS, the number of observations used in 
the analysis of the b” bootstrap replicate, n"’, is equal to 
the number of units k € s for which w,’ > 0. Such 
software packages may use n”’ — r instead of n — r when 
computing the bootstrap statistic 7(s, w.’; HB,,,). One 
must thus make sure that » — r is used and, if not, that the 
bootstrap statistic computed from these packages is properly 
adjusted before applying the proposed bootstrap test 
procedure. One way of avoiding this problem is to add a 
very small positive value (e.g., 1 x 107'° ) to each bootstrap 


29 


weight w,’, for k € s, so that no observation is excluded 
from the computation of 7 (s, w,’; HB,,.). 


Remark 2: Let us define the bootstrap statistic 7,(s, w.’; 0) 
by replacing y, by e, = y, —x;B,, in 7 (s, w.’; 0), for 
each k € s. It is not difficult to show that 7,(s, w.’; 0) = 
i(s, w.’; HB,,,) so that our bootstrap procedure can be 
implemented using either 7,(s, w.’; 0) or 7 (s,w.’; HB,,, ) 
when a linear regression model is used. The former may 
sometimes be more convenient with some software 
packages. This was the case in our simulation study since 
the use of 7,(s, w,’; 0) allowed us to get rid of manually 
typing the values of HB... for each selected sample. An 
informal explanation for the equality 7(s, w.’; 0) = 
i(s, w.’; HB,.) can be obtained by treating B,. as a 
fixed quantity, which is actually the case under the bootstrap 
distribution. The bootstrap statistic 7(s, w.’; HB,,) can 
thus be interpreted as a statistic aiming at testing the null 
hypothesis Hj: HB = HB,,. or, alternatively, Hj: Hy = 0, 
where y = B - Be Still assuming that B,,. is fixed, we 
can rewrite our linear model E,(y,|X,) = x,B as 
E,, (e, |Xy) = X,Y . These observations seem to imply that 
using the bootstrap statistic 7,(s, w,’; 0) is equivalent to 
using 7 (s, w.’; Hf,,..), which is indeed true. 


Remark 3: We have already mentioned that the WEIGHT 
statement is necessary to obtain a weighted statistic if the 
proposed bootstrap test procedure is implemented using the 
procedure REG of SAS. Also, the TEST statement is 
necessary to request the desired statistics to be produced and 
the “ODS OUTPUT TESTANOVA =” statement to save 
these requested statistics in a SAS dataset specified by the 
user. 


5. Some alternative procedures 
for linear regression 


‘In this section, we briefly describe some test procedures 
in the context of linear regression exposed in section 4; 
namely, two naive procedures that are sometimes used in 
practice as well as specific implementations of the Rao- 
Scott, Wald and Bonferroni procedures. They will all be 
evaluated in the simulation study in section 6. 

The Bonferroni, Wald and Rao-Scott procedures, 
described in sections 5.2, 5.3 and 5.4 respectively, all need 
an mp-consistent estimator Nid (B,,,.) of iat (B,,,)- In the 
simulation study in section 6, we have used the bootstrap 
variance estimator 


: (5.1) 
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It is worth noting that the validity of assumption 5 is thus 
not only required for our proposed bootstrap method but 
also for the Bonferroni, Wald and Rao-Scott methods. 


5.1 Two naive procedures 


The weighted version of the naive procedure consists of 
using the statistic 7 (s, w,; ¢) in (4.2), which is compared 
to the upper tail of the distribution Fy ,_,. The unweighted 
version uses the statistic f(s, 1,; ¢), which is again 
compared to the upper tail of the distribution F, ,_,. Both 
procedures are not expected to work well under informative 
sampling but are still often used in practice, especially the 
weighted version. Note that if sampling is not informative, 
the unweighted version, that ignores the sampling design, 
leads to a simple, valid and reasonably powerful test. 


5.2 The Bonferroni procedure 


The Bonferroni procedure was studied by Kom and 
Graubard (1990). It is simple to use and was shown to work 
well in their empirical study. To describe this procedure, let 
H/, represent the q" row of H and c, the gq" element of 
c. Then, compute the Q weighted statistics 


A iuhSarC, )) A Bos = 6) = c4) ; 
dhs ag Sood celled Fh 

The largest statistic Pea cy) yeforeg:=1,! 14 °Oys 
compared to the upper tail of the distribution F, , with a 
revised significance level a/Q instead of a. The number 
of degrees of freedom d is equal to the number of sampled 
primary sampling units minus the number of strata. Note 
that this procedure depends in general on the model 
parametrization used. 


(5.2) 


5.3 WALD F-procedure 


An F-version of the standard Wald chi-square statistic, 
with adjusted denominator degrees of freedom as proposed 
by Fellegi (1980), can be defined as 


i” (s;c) = 
dé r © hobs F 
sega ili - ¢) (AV, (B,,) 1) (HB,,, - ©). (53) 


The statistic 7” (s;c) is compared to the upper tail of the 
distribution Fp 4_9,;. This procedure is implemented in the 
software package SUDAAN (Research Triangle Institute 
2004). 


5.4 Rao-Scott F-procedure 


Another procedure consists of using an F-version (see 
Rao and Thomas 2003) of the second-order adjusted chi- 
square statistic of Rao and Scott (1981), which is based on 
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Satterthwaite’s correction for the number of degrees of 
freedom. We use an adaptation of these authors’ method for 
linear regression, as implemented in the software package 
SUDAAN (Research Triangle Institute 2004). The statistic 
is defined as 


1 
A(1+a*)O* 


(HB,,, — c)'(HV x5 (B,,,) H’)' (HB, - ©), (5.4) 


where V.,<(f,,,) is an estimator of the variance-covariance 
matrix of B,,. under a simple random sampling design, 1 is 
the average of the eigenvalues of the generalized design 
effect matrix [Vsp5(B,,, )] Vc (B,,,), a is the coefficient of 
variation of these eigenvalues and Q° = Q/(1+ a’). The 
Rao-Scott F-statistic 7*°(s;c¢) is compared to the upper tail 
of the distribution F’ one 


i (s:¢) = 


6. Simulation study 


We performed a simulation study to investigate the level 
and power of the above test procedures in the case of 
informative and non-informative sampling. In sections 6.1 
and 6.2, we describe the population and sample creation 
respectively. We then define the null hypotheses to be tested 
in section 6.3, describe the methods evaluated in section 6.4 
and present simulation results in section 6.5. 


6.1 Generation of the populations 


We generated four populations of N =10,000 units. 
First, a categorical variable v, was generated independently 
for each population unit k so that v, = i, for i = 1, ..., J, 
with probability P(v, = 7) =1/J, where J is the number 
of categories of v,, which was set equal to 5. The 
dependent variable y was generated as 


(I +1) 


y, =a,+, ( - +o9,, (6.1) 
where , ~ N(0, 1), a, =10 and o=3. The four 
populations that we generated only differ in the choice of 
a,, which controls the correlation between y and v. We 
considered a, = 0, 0.25, 0.50 and 0.75. 


6.2 Generation of samples and bootstrap weights 


From each of the above four populations, 5,000 stratified 
simple random samples of size 100 were selected without 
replacement under two different stratification scenarios 
aimed at simulating both informative and non-informative 
sampling. In the case of non-informative sampling, the 
strata correspond exactly to the five categories of variable v 
defined above. In the case of informative sampling, the 
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strata are defined by the cross-classification of variable v 
and another categorical variable z that depends on the 
random error term o@, in (6.1). For each population unit 
k, variable z was created as follows: z, = 1, if og, > 0, 
and z, = 2, otherwise. This leads to 10 strata in the 
informative case that are constructed by crossing the five 
categories of v with the two categories of z. Each of the 10 
informative strata contains about 1,000 population units 
while each of the 5 non-informative strata contains about 
2,000 population units. 

Furthermore, two different stratum allocation schemes 
were used. The scheme, SCHEME UNEQUAL, allocates 
the 100 sample units among the strata in the following way: 


Table 1 
Sample sizes for SCHEME_UNEQUAL 


v 
a 3 4 | 5 


Informative 
informative 


BERBER 


The second scheme, denoted SCHEME EQUAL, 
assigns the same number of units in each stratum as follows: 


Vv 


The two different schemes lead to very different sets of 
survey weights. The weights resulting from _ the 
SCHEME UNEQUAL allocation are much more variable 
than those from SCHEME EQUAL. Note that we simply 
defined the survey weight w, as the inverse of the selection 
probability of unit k. 

Finally, for each selected sample, 500 design-based 
bootstrap weights were calculated for each sampled unit, as 
described in Rao etal. (1992), among others. In our 
implementation of this methodology, each bootstrap sample 
was selected with replacement by stratified simple random 
sampling with n, —1 draws from the n, sample units in 
stratum h. This methodology takes the sampling -design 
variability into account (with a slight overestimation of the 
design variance due to assuming with-replacement 
sampling) but ignores the model variability. This is 
acceptable since the overall sampling fraction (1/100) is 
small. 


Table 2 
Sample sizes for SCHEME EQUAL 


informative 


o 
yas 
3 
E 
Ce 
as 
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6.3 Null hypotheses 


For each selected sample, we modeled y, as a function 
of v, using an analysis of variance model. More 
specifically, we defined indicator variables 


1 
xin. 0 


fof Cha de and fitted. “the. linear . model 
y, = Bo + Xia Bx, +£, using the weighted  least- 
squares technique, where ¢, is a random error term with 
mean 0 and constant variance. We considered testing the 
following two null hypotheses: 


if vy, = i, 


otherwise, 


TESTI: H,: B, = 0 
EST 2 ds Dis B> c= emia Diep SO. 


Note that both null hypotheses are true for the population 
with a, = 0 while they are false for the other populations. 
The latter three populations are used to assess the power of 
the different test procedures under study. 


6.4 Test methods 


For each selected sample, we tested the above two null 
hypotheses using five different methods: the proposed 
bootstrap method, the naive method (both unweighted and 
weighted versions) described in section 5.1, the Bonferroni 
method described in section 5.2, the Wald F method 
described in section 5.3 and the Rao-Scott F method 
described in section 5.4. Results for the naive method are 
standard output in the software SAS whereas the Wald and 
Rao-Scott F-statistics are standard output in the SUDAAN 
statistical software, version 9. The Bonferroni statistics (5.2) 
are also obtained through SUDAAN. The proposed method 
is programmed in the statistical software SAS, version 8. 

In addition, we also performed the simulation study using 
a linearized variance estimator in the Wald, Rao-Scott and 
Bonferroni methods instead of the bootstrap variance 
estimator (5.1). Rejection rates obtained using the linearized 
variance estimator were slightly lower but quite similar to 
those obtained using (5.1). Given this observation and that 
our focus is on bootstrap methods, we neither show nor 
discuss these additional results in the next section. 


6.5 Simulation results «— 


For each population, stratification scenario, allocation 
scheme, null hypothesis and method, we calculated the 
rejection rate in percentage over the 5,000 selected samples 
(using a 5% significance level). Results are given below in 
tables 3A, 3B, 4A and 4B. The results are more striking and 
more interesting for the null hypothesis TEST2 than the null 
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hypothesis TEST 1. We will thus focus our discussion of the 
results on the former. 

Tables 3A and 3B contain the results in the case of 
informative sampling, which is of more interest to us. Let us 
discuss first results in table 3A for SCHEME UNEQUAL. 
Both naive methods perform poorly as they do not properly 
exploit sampling design information. On the one hand, the 
unweighted version is definitely too liberal as its rejection 
rate is far above 5% under the null hypothesis. On the other 
hand, the weighted version is too conservative and 
significantly lacks power when compared to other methods. 
The Wald method is too liberal with a rejection rate of 
15.8% when H, is true. The simple Bonferroni method 
improves the situation although it is still too liberal with a 
rejection rate of 11.4% when H, is true. This result is 
somewhat surprising as the Bonferroni method is known to 
be (asymptotically) conservative. A referee suggested that 
we consider an improved Bonferroni method such as that 
developed by Benjamini and Hochberg (1995). In this 
simulation study, such a method would not help as it always 
rejects more often than the standard Bonferroni method. The 
Rao-Scott method significantly outperforms the Wald and 
Bonferroni methods under the null hypothesis with a 
rejection rate of 6.8%. The proposed bootstrap method is 
comparable to the proven but more complicated Rao-Scott 
method with perhaps even a slight improvement in the level 
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with a rejection rate of 6.2% when H, is true. However, the 
Rao-Scott method is slightly more powerful than the 
proposed bootstrap method. 

Table 3B contains results under SCHEME EQUAL in 
the informative sampling scenario. Here, the weighted and 
unweighted versions of the naive method yield similar 
results since the variability of the survey weights is quite 
small. Even in this case, the naive method is definitely too 
conservative, which results in an extremely low power. All 
other methods are comparable both in terms of level (H, 
true) and power (H, false) although the Wald method is 
still slightly too liberal compared to the Bonferroni, Rao- 
Scott and proposed bootstrap methods with a rejection rate 
of 7.9% when H, is true. 

Tables 4A and 4B contain the results in the case of non- 
informative sampling. Again, let us discuss first results in 
table 4A for SCHEME UNEQUAL. As expected, the naive 
unweighted method performs well here while the naive 
weighted method becomes too liberal with a rejection rate 
of 12.8% when H, is true. In terms of the level, the 
proposed method is competitive to the naive unweighted 
method and even slightly conservative. It outperforms the 
Wald method and is slightly better than the Bonferroni and 
Rao-Scott methods. Its power is however slightly less than 
these latter two competitors but still acceptable. 


Table 3A 
Rejection rates at the 5% significance level under SCHEME UNEQUAL and informative 
sampling 
SCHEME_UNEQUAL Informative Sampling 

Ho TRUE Ho FALSE Ho FALSE Ho FALSE 

a, =90 a, = 0.25 a, =90.50 a, =90.75 

Method Test! Test2 Testl Test2 Testi Test2 Testl Test2 
Naive Unweighted 37:50 f00.01¢ 85.3 “100.08 59838™ 100:0 - 1000.7 100:0 
Naive Weighted 1.7 0.4 14:5 46 58.0 33.6 90.3 78.6 
Wald 8.0 15.8 30.9 37 Ame TES I39 3ch 95.4 
Rao-Scott 8.0 6.8 30.9 PA 0 paoaay I. 61.7 301 91.8 
Bonferroni 8.0 11.4 30.9 B20. 1s 68.8 93.1 S19 
Proposed Bootstrap 7.4 6.2 29.4 Is at abel Ue 9:9 92.8 91.0 
Table 3B 
Rejection rates at the 5% significance level under SCHEME EQUAL and informative 
sampling 
SCHEME_EQUAL Informative Sampling 

Ho TRUE Ho FALSE Ho FALSE Ho FALSE 

a, =0 a, =90.25 a, = 0.56 a, = 90.75 

Method Test] Test2 Testi Test2 Testi Test2 Testl Test2 
Naive Unweighted 0.1 0.0 6.7 1 16.5 8 Op 
Naive Weighted 0.1 0.0 6.3 0:3"~, 20:8 18.2 97.0 81.4 
Wald 5.8 13,., 43.6 SN Pelee © 905 92.3 99:9 ==100 6 
Rao-Scott 5.8 o.25 43.6 220i D7 90.4 99.9 99.9 
Bonferroni 5.8 6.2 43.6 3356 1 O3aF 88.6 99.9 99.8 
Proposed Bootstrap 233 5.1 42.3 BRO stOR6 89.6 99.9 99.9 
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Table 4A 
Rejection rates at the 5% significance level under SCHEME UNEQUAL and non- 
informative sampling 
SCHEME_UNEQUAL Non-Informative Sampling 
Ho TRUE Ho FALSE Ho FALSE Ho FALSE 
Q, = 0 Q, = 0.25 Q; = 0.50 Q, = 0.75 
Method Testl Test2 Testl1 Test2 Testi Test2 Testl Test2 
Naive Unweighted 4.2 4g foe 13:5 12 Ah B99 34.6 71.8 70.5 
Naive Weighted 11.4 12.8 24.6 Zo0 ” 6.6 SOM 83.8 81.2 
Wald 7.6 8.6 16.8 17.8 «42.9 42.6 dD 76.2 
Rao-Scott 7.6 64 16.8 12:3. 42.9 32 125 EP) 
Bonferroni 7.6 TAs LO:8 16.5 42.9 42.1 (P=) ORO 
Proposed Bootstrap 6.3 45 144 Oe 385 26.4 68.2 56.4 
Table 4B 
Rejection rates at the 5% significance level under SCHEME EQUAL and non-informative 
sampling 
SCHEME_ EQUAL Non-Informative Sampling 
Ho TRUE Ho FALSE Ho FALSE Ho FALSE 
a, =90 a, = 0.25 a, = 0.50 a, = 0.75 
Method Testl Test2 Testi Test2 Testl Test2 Testl Test2 
Naive Unweighted 4.9 4.5 172 12.4 54.3 42.2 88.2 Sly 
Naive Weighted 5.0 4.5 17.4 | Pee ade ST 42.7 88.3 81.9 
Wald 5 6.9 18.8 16.3 56.6 48.9 88.9 85.0 
Rao-Scott Sur 5.0 18.8 IS” 96.6 49.2 88.9 82.6 
Bonferroni Suh 5.4 18.8 ose 0:6 45.1 88.9 81.8 
Proposed Bootstrap 5.0 23 16.4 Oma .2 36.5 86.8 77.6 
Table 4B contains results under SCHEME EQUAL in hen 5 i weg, i we of Ine 
é ¢ : : m ejection rates at the 0 Significance level under 
the non-informative sampling scenario. In this table, the SCHEME_UNEQUAL 
methods do not appear to differ drastically. As expected, the - 

" ; 2 : SCHEME_UNEQUAL Informative Non- 
naive method (both weighted and unweighted versions) srormative 
performs well although it did not outperform the Rao-Scott Ho TRUE Ho TRUE 
and Bonferroni methods in this simulation study. The a, =0 a, = 
proposed method is still slightly conservative in this non- Method __ peter ee tape Nest es 
inf; : : ay lightly 1 en th Naive Unweighted 100.0 100.0 aut 3.8 
Inrormative SCcenano an as Sil y lesS power tan we Naive Weighted 13 0.7 93 10.5 
other methods. Wald 4.6 4.5 32 4.1 

To investigate the effect of large samples on the test | Rao-Scott_ 4.6 3.8 3.2 3.8 
procedures, we also performed some simulations with Eon ae fe ve os 

: Proposed Bootstrap 4.4 3:6 28 3.8 


sample sizes that are ten times larger than in the original 
setup, as suggested by one reviewer. That is, we considered 
a population size of 100,000 and selected 1,000 samples of 
size 1,000 thus deliberately keeping the same small 
sampling fraction. From this setup, we obtained results 
when H, is true, shown in table 5, for both informative and 
non-informative sampling under unequal stratum allocation. 
As expected, all the methods other than the naive ones have 
similar rejection rates that are indeed slightly lower than 
5%. This illustrates that the differences between the 
methods become less important as the sample size increases. 


Overall, our proposed bootstrap method was the best in 
terms of the level, followed closely by the Rao-Scott method. 
It gave somewhat conservative results in the non-informative 
sampling scenarios. This was accompanied by a slight loss of 
power. The Rao-Scott method is a good alternative if users 
have access to an appropriate software package. The 
Bonferroni method is simple to use but may be too liberal and 
the Wald method is even worse. The naive methods may 
have serious deficiencies, either in the level or in the power, 
although the naive unweighted method is viable if one is 
reasonably sure that sampling is not informative. 
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7. Summary and discussion 


We have proposed a general and simple bootstrap 
procedure for testing hypotheses from survey data, which 
could also be applied outside the survey sampling field. Our 
procedure uses classical model-based test statistics and is 
thus easy to implement for analysts using classical software 
packages. We have shown in a simulation study that it 
performed well in the context of a linear regression model. 
These good results are encouraging and may suggest that 
our proposed bootstrap procedure could be useful with other 
more complicated models and other statistics. The idea 
could also be easily adapted for the construction of 
bootstrap confidence intervals. 

One could also consider bootstrapping an asymptotically 
pivotal statistic such as the Rao-Scott statistic (5.4). This 
would however involve double bootstrapping if Vc (B,,. ) 
is estimated using the bootstrap technique as in (5.1). 
Double bootstrapping requires generating another set of 
bootstrap replicates for each initial bootstrap replicate. 
Although better test procedures could potentially be 
obtained, double bootstrapping may not be convenient for 
analysts. By focusing on simpler statistics that do not 
involve the bootstrap technique, our test procedure avoids 
double bootstrapping and remains simple. 

The properties of our method depend not only on the 
choice of the test statistic but also on the construction of the 
bootstrap weights. Typically, bootstrap weights capture the 
first two design moments of the sampling error, which 
should be sufficient in most cases to satisfy our bootstrap 
assumptions 3, 4 and 5. Bootstrap weights that also capture 
the third design moment could perhaps be useful for 
improving the level accuracy of the bootstrap test. This 
needs further investigation. Finally, as already pointed out in 
section 3, standard design-based bootstrap weights satisfy 
assumption 5 only when the overall sampling fraction is 
negligible so that the model portion of the total variance 
(3.1) is negligible. Research is needed to develop proper 
bootstrap weights, when a non-negligible sampling fraction 
is used, that capture both the model and the design portions 
of the total variance. 
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Appendix 
Proof of result 1 


From assumption 1, we can easily see that 


Jn (HB,, - HB) —“2—> N(0,HEH’).  (A.1) 


Using a standard result on quadratic forms (e.g., Seber 
1984, page 540) and equation (A.1), we obtain 


Q 
n(HB,,,- HB)’A '(HB,,,-HB) —W— ))A,Q,, (A.2) 


where 4,, for g =1, .... Q, are the eigenvalues of A = 
(A7')(HZH’) and Q., are independent chi-square random 
variables with one degree of freedom. Therefore, from (A.2) 
and assumption 2, we have 


i (s,w,; HB) = 


, ; ‘ Q 
(HB,,,— HB)’{A(s, w, )}-"(HB,,,— HB) —* >) 1.,Q,,. 


q=1 
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Bayesian methods for an incomplete two-way contingency table with 
application to the Ohio (Buckeye State) Polls 


Bo-Seung Choi, Jai Won Choi and Yousung Park ' 


Abstract 


We use a Bayesian method to resolve the boundary solution problem of the maximum likelihood (ML) estimate in an 
incomplete two-way contingency table, using a loglinear model and Dirichlet priors. We compare five Dirichlet priors in 
estimating multinomial cell probabilities under nonignorable nonresponse. Three priors among them have been used for an 
incomplete one-way table, while the remaining two new priors are newly proposed to reflect the difference in the response 
patterns between respondents and the undecided. The Bayesian estimates with the previous three priors do not always 
perform better than ML estimates unlike previous studies, whereas the two new priors perform better than both the previous 
three priors and the ML estimates whenever a boundary solution occurs. We use four sets of data from the 1998 Ohio state 
polls to illustrate how to use and interpret estimation results for the elections. We use simulation studies to compare 
performance of the five Bayesian estimates under nonignorable nonresponse. 


Key Words: Bayesian analysis; Nonignorable nonresponse; Contingency table; Boundary solution; EM algorithm. 


1. Introduction 


The problem of nonresponse is common in most surveys 
becoming a serious issue as the nonresponse rate increases 
(De Heer 1999; Groves and Couper 1998). When survey 
data is summarized in a two-way contingency table, the 
table includes fully classified counts, partially classified 
counts (i.e., item nonresponse), and unclassified counts (i.e., 
unit nonresponse). For example, in the Ohio (Buckeye 
State) Poll (BSP) (Chen and Stasny 2003), one category 
involves the voting preference (candidates A,B,C, or unde- 
cided) and the other category is the likelihood of voting 
(likely to vote, not likely to vote, and undecided). First 
supplemental margin contains data only on the voting 
preference, second contains data only on the likelihood of 
voting, and third is only the number of unit nonresponses 
(both responses unknown). Our interest is to incorporate 
these missing observations into estimating the true support 
for each candidate and to present Bayesian models to predict 
the winner. 

In some surveys, the undecided answers are treated as a 
valid response category when the respondents do not have 
strong preference for a candidate and voting intention 
(Smith 1984; Rubin, Stern and Vehovar 1995). Many 
studies, however, have shown that the voting behavior of 
the undecided voters can have a significant impact on the 
final result and that by considering these undecided voters, 
the accuracy of election forecasting can be improved (Perry 
1979; Fenwick, Wiseman, Becker and Heiman 1982; Myers 
and O’Connor 1983; Kim 1995; Chen and Stasny 2003; 
Martin, Traugott and Kennedy 2005). Perry (1979), among 


them, showed that the undecided percentage in a poll is 
likely to be greater than the true percentage by presenting an 
empirical evidence using a secret ballot approach. Kim 
(1995) also indicated that these undecided voters are critical, 
especially in cases where the number of undecided voters is 
greater than the gap between the two leading runners in an 
election race. Three of our empirical studies in Section 3 
belong to this critical case. Fenwick ef al. (1982) and Kim 
(1995) applied a discriminant analysis to the October 1980 
poll data in Massachusetts and the 1992 USA presidential 
election, from which they allocated the undecided voters to 
candidates to show that undecided voters generally do not 
vote in the same proportions as their decided counterparts. 
When the focus is on the candidate the undecided voter may 
vote for, undecided responses are better treated as missing 
data (Myers and O’Connor 1983). As indicated in Flannelly, 
Flannelly and McLeod (2000) and Lau (1994), the 
forecasting error for the actual election results increases as 
the rate of undecided voters increases. To overcome this 
problem, Monterola, Lim, Garcia and Saloma (2001) 
applied a neural network approach to classify undecided 
voters in a public opinion survey. Smith, Skinner and Clarke 
(1999) and Molenberghs, Kenward and Goetghebeur (2001) 
utilized model based imputation methods for the 1992 
British General Election Panel Survey and the 1991 
Slovenian plebiscite public opinion survey. Because our 
main goal is to obtain more accurate forecasts by allocating 
undecided voters to proper cell, we treat undecided voters as 
missing observations in the same way as these researchers 
handled them. 
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Nonresponse (or undecided, equivalently) can be distin- 
guished by three types of nonresponses (Little and Rubin 
2002, page 11): missing completely at random (MCAR) 
means that the probability of a nonresponse on a variable of 
interest is independent of all survey variables including 
itself; missing at random (MAR) means that the probability 
of a nonresponse depends only on the observed data; 
missing not at random (MNAR) means that the probability 
of nonresponse depends on the unobserved values. Models 
for MCAR or MAR are called ignorable nonresponse 
models while models for MNAR are called nonignorable. 
For example, in a pre-election survey, if the respondents do 
not answer with their preference of a candidate, although 
they support a particular candidate, the pattern for candidate 
preference can be different between the respondents and 
nonrespondents. Then, the nonresponse mechanism is 
nonignorable. When data is assumed to be MCAR, the 
effect of nonresponse can be removed in likelihood 
inference (Little and Rubin 2002, page 11). However, when 
the nonrespondents follow a response pattern different from 
that of the respondents, discarding nonresponses or mis- 
specifying the nonresponse mechanism leads to larger 
variances and biases in estimation (Chen 1972; Park and 
Brown 1994). 

When nonresponse is nonignorable in contingency 
tables, ML estimation often yields boundary solutions 
where the probability of nonresponse is estimated to be 
zero in some cells. These boundary solutions often provide 
a local maximum of the likelihood function. In this case, 
the maximum likelihood (ML) estimates of the loglinear 
model parameters cannot have a unique solution and 
usually have large standard deviations (see Section 4 or 
Baker, Rosenberger and Dersimonian (1992) and Park and 
Brown (1994) for more detailed discussions). 

The conditions where the ML estimate falls on the 
boundary solution have been proposed in a one-way 
contingency table (Baker and Laird 1988; Michiels and 
Molenbergs 1997). The geometric explanation for the 
boundary solution of the ML estimate was presented (Smith 
etal. 1999; Clark 2002). Baker etal. (1992) presented a 
sufficient and necessary condition under which the ML 
estimate can have a boundary solution in a two-way 
contingency table. 

To overcome such a boundary problem in the ML 
estimate under the existence of nonignorable nonresponses, 
Park and Brown (1994) and Park (1998) proposed Bayesian 
approach using empirical priors based only on respondent 
information. Clogg, Rubin, Schenker and Schultz (1991) 
used constant prior for an incomplete one-way contingency 
table. Although they showed that, under nonignorable 
nonresponse, Bayesian methods provided smaller mean 
squared errors (MSE) than ML estimate in estimating cell 
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expectations, our simulation study shows that this is 
generally not true in an incomplete two-way contingency 
table. Thus, we present two Bayesian models whose priors 
depend on information from both respondents and 
undecided. We, then, apply each to analyze incomplete two- 
way contingency table. An extension to a multi-way table is 
straightforward. We can easily apply this extension to 
weighted data from stratified or cluster sampling using 
appropriate covariates (see Section 2.2). 

The remainder of this paper is divided into four sections. 
In Section 2, we consider Bayesian models with five 
different priors and present a generalized expectation 
maximization (EM) algorithm to estimate cell probabilities. 
In Section 3, we apply the Bayesian models to four 
empirical data sets from the Buckeye State Poll (BSP) and 
compare the Bayesian estimates with the ML estimate and 
the actual election results. In Section 4, we use simulation 
studies to compare MSEs and biases of the Bayesian 
estimates from different missing percentages and response 
patterns of the respondents and nonrespondents. In this 
section, we also calculate the coverage probability to 
examine the performance of the Bayesian estimates. Section 
5 includes some concluding remarks. 


2. Bayesian models 


We discuss five Bayesian estimates to accommodate 
nonignorable nonresponse in an incomplete two-way 
contingency table. We present an EM algorithm to tackle 
the nonresponse problem in a two-way contingency table in 
Section 2.1. Then, in Section 2.2, we specify five priors and 
extend our approach to a multi-way contingency table. 

Let X, and X, be response variables indexed by J and 
J categories, respectively, in a two-way contingency table. 
We also let R,= 1 when X, is observed and R,= 2 when 
X, is missing. Similarly, R,= 1 when X, is observed and 
R,= 2 when X, is missing. Then the full array of X,, 
X,, R, and R, constructs a J x J x 2x2 contingency 
table in which we have completely classified counts, 
partially classified counts, and unclassified counts. To 
distinguish these three types of observations, let y,,, be the 
count belonging to the i" category of X vacthe j'" category 
of X,, the k" value of R,, and the /" value of R,. Thus, 
Ys 18 used for the completely classified counts, y,,,, and 
¥,j2, for the respective column and row supplemental 
margins, and y,,,, for the unclassified counts. We assume 
a multinomial distribution for these three types of 
observations to have the following log likelihood: 


ie papbeyit ‘ log(T 11) ‘i yay) - log(7;,:2) 
re i i 


+ Din ; log (1, jo) + Vi422 ° log(t,,29) (1) 
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where Ty = Pr[X, = i X, 
N= Di, kt Yin is fixed. 

Since this likelihood function involves more parameters 
than degrees of freedom available for estimation, we link 
7,4, to relevant covariates using a loglinear function. Since 
no explanatory variable is available, we do not use any 
explanatory variables. However, the loglinear model can 
easily incorporate explanatory variables in the same way as 
it incorporates the categorical variables (see Baker and Laird 
1988 and Park and Brown 1994 for details). 

A nonignorable nonresponse model for all of the 
variables X,, X,, R,, and R, is defined by 


= j, R) =k Ry = 1) and 


log(m)) = By + Bx, + Bx, + Ba, ae 

+ Bin + Bee, + Bix, + Ber, 
for i=litveia Palio he nm and /=1,2 () 
where Mis = N-,,, is the expected cell count for the 
(i, j,k, )" category and the sum of each  -term over any 


of its respective super script(s) 1s zero. 

This loglinear model is saturated since the number of 
parameters is exactly the same as the number of cells 
observed from the incomplete two-way contingency table. 
This model is also a nonignorable nonresponse model 
because of the interaction terms between X, and R, and 
between X, and R,, implying that the nonresponse of 
each response variable depends on its own status. The 
loglinear model is a tool frequently used for analyzing 
incomplete contingency tables with nonignorable non- 
responses. Let p be the number of parameters (i.e., B ) to 
be estimated. We introduce the p x1 design vector z,,, to 
indicate the affiliation of the observation belonging to the 
(i, j,k, 1)" category. Then the loglinear model given in (2) 
can be rewritten as 


logm = ZB (3) 


where the J x J x 2 x 2 vector m is the cell expectation 
and f is the vector representation of the Bs. To avoid a 
boundary solution of the ML estimate in model (2), we 
impose Dirichlet priors to the cell probabilities (1 
Toi» Ty29) AS given by 


yl? Tii129 


Cs) 


5 a 
ple» nil? P rx?! h ij22 
l I Ist Mina ~ Myo) ~ Myr (4) 


where the hyper parameters, the 6,,,S are specified in 
Section 2.2. These Dirichlet priors produce an explicit and 
convenient form of a posterior distribution because they are 
conjugated to a multinomial distribution (Clogg ef al. 1991; 
Park and Brown 1994; Forster and Smith 1998). Together 
with (3), the multinomial distribution of (1) for 
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observations, and the prior distribution (4), we have the 
following log posterior distribution: 


bos = din “G1 B) 

= 2d ey log 2, xP ; B)) 

+ Dynan loa(Terplea  B) 

- Linz loa 2. Pn B)) 
. LY - log LEXP Car B)) 
= LY. he loa 2. 


ih Meteyy & log( 2. DexP@san B)) 


EXP(Z jx ° B)) 


Taf shes 


Acre log De, EXPZijus B)) 


i,j,kl 


a » Sie 5 Ca - B) 


i, kyl 
¥ De 9 it Tog » EXP(Z 4) ‘B) (5) 
i, jl ik! 


Equation (5) is rather complex and thus we use the EM 
algorithm to estimate the parameters (i.e., B ). 


2.1 The EM algorithm 


We maximize the posterior distribution given in (5) over 
the parameter B using the generalized expectation maxi- 
mization (GEM) algorithm (Dempster, Laird and Rubin 
1977) with the following E and M steps. 

E-step: Using augmented yj,., Yj. and yj) _ for 
b= Lead eand j= 1, 2, J) *the*posterior(5)° canbe 
written as 


‘ee = 220m + 8y,) log(m 1) 
% SF i + 8417) log(t2) 
aay 
z err + 84) log(tj21) 
ineay 
un PIS (Vio + S22) log(t;29). (6) 
is 
To determine the expected augmented log posterior in 
(6), we average over the missing counts ),,>, Yj), and 


ie: : ld 
Vij22 conditioning on the current parameter estimates, Tass 
and the marginal sums y,,., Y, 21 and y,,59: 
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E old [hese 
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Tons Yisi2) + Sina) 108 (Tyq2) 


fe Dd EnalYin2 
es, 


Tas Vs 721 ] si 821) ; log(%,j>1) 


+ DY Eales 
‘i | 


Teo» Vi422] ae 83,22) . log(%,;7)- (7) 


+ Dd Enialyi22 


Since Yj, Vjor and yy are multinomial random 
variates conditioned on the respective marginal sum y,,,,, 
Y.jo» and y,,,, the conditional expectations in the 
equation (7) are given by 


old 
old vas 12 
Eva Vir | Tiki Visi2) = el? old? 
i+12 
old 
721 
Y+j21 old ? 
+j21 


Id by 
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and 


old 

ij22 
Y 4422 old 

++22 


Id ‘4 
Eva Vin | Tiel» Ys422) + 


old old 
where mj, = N > Typ 


M-step: In this step, we maximize the expected log 


posterior (7) using si Beek observations Yin = ies 
ie) ce) ce) ce) 
Sip Ya = Visi2 Miz [M17 + Bi» YVjo1= Vo jar Myai (Ma + 


Sian and Vir = Varn Myo Alaa gf 8522 We impose the 
constraints on these pseudo observations so that their 
marginal sums are the same as the corresponding marginal 
sums of observations: 9, 44)=Vssiy Viers = Vise Veja = 
Ysjov and Y,,5.=¥Y,,29. Under these constraints, the 
pseudo observations are now 


= V4) = a 

Via ae LO ela ee 
Th ef + 8.44 
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RST Te hssiB Din or an 

Vijkt ~ y 

~ +j21 

Veoy ——=—_ for k = 2 and 1 = 1 
ey) Si a 8, 521 

Pyle eM = hf Eien arid SD. 


4422 7 8,499 


Statistics Canada, Catalogue No. 12-001-X 


Then, the expected log posterior function (7) becomes 


1 EE 
+ Ld Ynn 
+ Dyin 
+ Davina 


. log(;,,) 


Eoallazss 
‘ log(,7) 
log(t,>1) 
- log (9). 


This equation has the same form as the likelihood obtained 
from a four-way contingency table with fully observed cell 
counts ee s. Thus, using the iterative re-weighted least 
squares method (Agresti 2002, page 342), we obtain the 
maximum posterior estimator (MPE) of B as follows: 


pe ZV, Ze ZV Ey 


where y“ has element y{7, = log m\2, + (Yjx: — yy) / My 
and V, = [diag(m”)]". We finally iterate these E and 
M-steps until a convergence criterion is achieved. The 
convergence criterion we use is ¢ < 10°, where ¢ is the 
difference between two consecutive log posterior functions. 

Let aes = Vip Yi+12? V+ jp Y+422) and ie oy 
(Vir Viv Vy) for i =1,...,/ and j =1,...,J be the 
observed count vector aud the missing count vector, 
respectively. Then the log posterior distribution (5) can be 
written as 


= = 1p | ) Se) Tae l(p |e obs? Tis) 
— log f Ynis | Yoos> B)- (8) 


By taking differentiation twice with respect to B, (8) 
yields 


O71(B | Me) a l(p ed obs? ie) 
ApoB’ pop” 
4; log Ate mis BYE obs? B) 
apap” 
= — Z"[diag(m) — mm’/N] Z 


+ Z"[diag(x) — mn’ ] ABZ, (9) 


where m is vector expression of cell probabilities 1,,, and 
A, B are given by 
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0 0 0 0 
m 
OuePding |e tle ae 0 0 
A= Viva. Siig Mise 
= ; i 
0 0 diag V+ j21 21 0 
Ysjr + 8,521 Mm, 521 
2 
0) 0 0 diag V4422 ij22 
View | O..57 ei 
and TM 122° 1222 M322 M122 M2222 2322 
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Here, to save the space and since there is no difficulty to 
extend for general i and j, B’, B*', and B” are illustrated 
only for J = 2 and J = 3: 


m m m 
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We observe that the observed data information 
2°1(B | Y¥,,,)/OBOB” is equal to the augmented data 
information minus the missing data information. As shown 
in Gelman, Carlin, Stern and Rubin (2004, page 103), the 
inverse of the observed data information evaluated at the 
MPE of B is the variance of the MPE of B. 


2.2 Specification of priors 


To complete the EM algorithm, we need to determine the 
hyper-parameters, 6,,,s. We set the sum of pniors 
Di, j,&1 Oj qual to the number of parameters involved in 
the loglinear model, p, as suggested by Clogg ef al. (1991). 
Under this constraint, we propose five types of priors as 
follows. We first allocate 5,,, for the MPE of m,, to 
shrink toward the MLE obtained under ignorable non- 
response. That is, we determine 6,,, depending only on the 
known response counts y,,, and call them respondent- 
driven priors. 

The first type of respondent-drven prior is, for all 
A= 1). ¢f and gjiaalie’. oJ, 


if Jian 
ijl 1 » O12 12 » O72) 
4 Vat) V4! 
Jil “i Siu 
+411 ++11 
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where, Vi." 228 View) Vee Ook 1,2 and P= 1, 2. 
The second type of respondent-driven prior gives no prior 
(i.e, no need of prior as described below) on 7,,, in the first 
type of priors. That is, the second type is the same as the 
first type except 6,,, = 0 for all i and j. In the case of a 
one-way contingency table (i.e. either X, or X, 1s fully 
observed without missing information) and y,,,. = 0, the 
first type is reduced to the priors used in Park (1998), 
whereas the second type is reduced to the priors used in 
Park and Brown (1994). These two types of respondent- 
driven priors may be too simplistic because the non- 
respondents are usually assumed to have a different 
response pattern from the respondents under a nonignorable 
nonresponse model. For example, the candidate preference 
of nonrespondents could be different from that of 
respondents in a pre-election survey. 

In order to define the third type of prior, denote m,,, as 
the MLE for m,,,. The closed form of m,,, can be obtained 
from Baker etal. (1992) where some m,,, could be zero 
because of boundary solutions. For example, when a 
supplemental column margin has a boundary solution in an 
incomplete 2 x 2 table, the MLEs are m4, = Y,11; 


— Yoana + V+j21) 


Maj "i Vote AAV ccd us m2 f Muni? 
where b, is the solution of 24-1415; = Yisi2s Mayr = , 
M2) = Pity 22, ™M, jo, = 9, 
Y2411 
and = My 59) = M17 Vi422/ Yasin. Therefore, these ML 


estimates accommodate both the information of respondents 
and nonrespondents, as well. The ML estimates can also be 
obtained from our EM algorithm in Section 2.1 by setting 


Table 1 
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Sil = 0 for all i, j,k and /. Using these ML estimates, 


we define the third type of prior as 
Si Vi {$24}, by, 
11 
aya ue yet ed 
V2 et ‘ I J 2° 
mM. 
6. {#2 et )b 
it . mM, 9) 1 | 2 
and (11) 
. sk _( Mir lg 
Syn Von (a i iM 2 


where V,, = p-m,,,,/m,,,, for k,/ = 1,2, and the term 
1/LJ is the constant prior of Clogg et al. (1991) to prevent 
possible boundary solutions for m,,,, ™,,, and mj», (also 
see the fifth prior below). Thus, we allocate the third prior of 
Six, for the MPE of m,,, to shrink toward the ML obtained 
under the nonignorable nonresponse, whereas the first prior 
is obtained under an ignorable nonresponse model. 

The fourth type of prior is defined by letting 5,,, = 0 in 
(11) as we did in obtaining the second type of prior from the 
first type. The last type of prior is from Clogg et al. (1991) 
defined as 


Byun = 0-82 =F «(71 5)s8ym = § (719): 
and (12) 
m= (7) 


These five types of priors are summarized in Table 1 and 
are compared in the next section using empirical data and 
simulation studies. 


Five types of priors 3,4, (sj, is MLE, J and J are the numbers of row and columns in a two-way table, and p is the 


number of parameters) 


dit dy12 dj21 8522 
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Up to this point, we have presented methods for a two- 
way in and yx) is aegned for the count ie the (i, 7) cell 
of the i” row and j™ column (ie, “oa = j), and 
indicator R, for a missing row ert R, ie a missing 
column (i.e, R, = k,R, = /). This can be easily extended 
to the 3-way table. Denote yi, to be the (i, j,k)" cell 
count for the three response variables (ie, X, = i, 
X, = j, and X, =k) and respective missing rows and 
COMMU MLC Re tig tle hee aie MANIC ne TL Ol 
l,m,n =1,2). Thus, /mn = 111 implies that all of the 
three variables are observed, /mn = 112 implies that X, 
and X, are observed but xX, is missing; similarly for 
Imn = 121,122, 211, 212, 221, 222; 1 is for observed and 
2 designates missing. Accordingly, the EM algorithm and 
priors for an incomplete three-way contingency table can be 
defined. The conditional expectation in the E-step for the 
(i, j,k)" cell with unknown information of k margin is 

old 
Ena Vina i2 | TV si) = rings 
+112 
Similarly, 
m00 


old = 
Ea ikN22 | Tsitimn> Yi+4122) Pn Are TS pein 
i++122 


and 
old 
ijk 222 
Via) ar old 
++4222 


old 2: 
Ena ijk 222 | Teistimn> Y++4222) Yr 


Other expectations and five types of priors can be similarly 
defined. 

The Buckeye state poll is a Random Digit Dialing 
(RDD). No modification is necessary for the Bayesian 
procedures if the RDD is strictly a self-weighting survey 
(Lavrakas 1993; Potthoff 1994). However, RDD is not 
always done by a self-weighting design. For example, a 
telephone sample comprises a sample of households, not 
persons. If one person is interviewed in a household, a 
weight should be superimposed on the response by the 


number of persons in the household. A weight is also. 


needed for the households with more than one telephone 
number. If an accurate estimate of the total number of 
households is available, stratification by region or state is 
possible and weighting must be considered in a 
comprehensive analysis. RDD was used in the 1998 Ohio 


election surveys. In this study, our method and models do 


not include weighting from stratification, clustering, and 
other factors leading to different probabilities of selection in 
a telephone survey. 

However, further extension can be made for such 
weighting. A simple extension below shows how to 
accommodate a typical stratification. In a three-way table, 
let X, be the third response variable indexed by h 
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(h = 1,..., H) that is assumed to be always observed. The 
H categories can be strata in a stratified sampling. Since 
X, is always observed, the corresponding missingness 
variable R, is equal to | and its observation can be denoted 
DY Viihim- Then, we can write the following log likelihood 
for each stratum h: 


Teel 1 
= Dy Yim OS(s) + SE Paes log(%;, m2) 
mya =I 


— 


WNC Ti lag ke fk Lh mn | X, = Al. 
Thus, the terminology X, used ie a three-way table acts as 
an indicator for strata. For each stratum h, the likelihood of 
(13) is exactly the same as that of a 2-way table. 

Then, a log linear model for the cell expectation 

Minim = Nj, * Tjhim CAN be defined in a similar way as in (2) 
Whete Nii (2 ci = Venn OL CoC 1, 2... it. A 
nonignorable nonresponse model is given by 


I 
log(™ nim) = Bos + Bua + Bip + Bra 
r ‘ 
hs Bryn ‘i By xh rT aa i BY ar (13) 


To avoid a boundary solution problem as in Section 2, we 


use the Dirichlet priors for 104i, 


HI Siihll i in Oi? 
Tim ° Mim ijh21 * ynr2 - 


Then, we follow exactly the same procedures as shown 
in Section 2 to estimate the cell iugdeuor Minim for each 
h = 1,2...,H. The estimate of the (i, 7)" cell expectation 
is 

2 H 
E(y,) = 2” LP ni 
where w, is the known weight for the fh" stratum and 
Minim 3S the My, evaluated at the MPE of B. For 
example, w, = N,/>,,N,, is for a stratified sample where 
N,, is the population size of the h" stratum. 

The . variance-covariance matrix of an approximation to 

the distribution of m_ is 


- Var Binz) (14) 


Of 2p 

where th is a vector expression of the cell estimates 

Mihi pire is the MPE of B and its variance Var(Bups) 

is given by the inverse of (9), and Om/op = N, x 
[diag(#%) — w#"]Z where 7 has 
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CXP(Z jrimPB PE) 


ke(i,j,hl,m) exp(Z,Bupe) 


Te him = Ttim( Bure ) = 3 


as its typical element. 


3. An application to a Buckeye State Poll 


In forecasting the winner in a poll, the accuracy of the 
poll often depends on how to handle undecided voters who 
are likely to vote but who have not yet decided their 
preference for a candidate. We compare the Bayesian 
estimates based on the five types of priors with the ML 
estimate using the Buckeye State Poll (BSP) conducted in 
1998 by the Center for Survey Research at Ohio State 
University. The BSP surveys produced incomplete two-way 
contingency tables with one category being candidate 
preference and the other category being the likelihood of 
voting in the November 1998 races for Ohio Governor, 
Attorney-General, Mayor of Columbus, and Treasurer. 
Table 2 summarizes these four polls and shows a substantial 
number of undecided voters. 

For comparison, we consider the following ignorable 
Model | and the two nonignorable nonresponse Model 2 
and Model 3. 


Model 1: log(m,,;) = By + By + Br, i Br, 


/ i kl 
dj Br, i Byax, + Brey 


Model 2: log(m,,;) = By + By _ Bry, + Br, ¥ Br, 


ik il ij Ml 
\ Br. “ Bre, + Brix, t Bary 


Table 2 
Observed data for BSP pre-election surveys 


Governor race 


Model 3: log(mj,) = By + By, + Bx, + Br, + Ba, 
il ik ij kl 
uy a a By r, 5 By. x, = Brak: 


Model | is missing completely at random, and cases with 
missing data can be ignorable in likelihood inferences. 
Model 2 and Model 3 are nonignorable where the 
probability of missing a variable depends on itself in Model 
2 while the probability in Model 3 depends on the other 
variable. Note that the ML estimates in Model | and Model 
3 are not on the boundary of the parameter space as shown 
by Baker etal. (1992). Moreover, since we found that, 
under Model 1 and Model 3, all of the five Bayesian 
estimates for the expected cell counts are not only fairly 
close to the ML estimate and their standard deviations are 
almost the same, we only present the ML estimates for 
Model | and Model 3. 

We denote the ML estimates under ignorable Model 1, 
nonignorable Model 2, and nonignorable Model 3 by 
IG\y,, NON2,,, and NON3,,, respectively. JG and 
NON stand for ignorable and nonignorable, respectively. 
We also let NON2?" be the Bayesian estimator using the 
i" type of priors under Model 2. That is, NON2?* uses 
the respondent-driven priors of (10) and NON25" is the 
same priors as NON2/" except for dj; = 0. Similarly, 
NON23" is given by (11) and NON22" is the same priors 
except for 6,,, = 0. NON22" is the Bayesian estimate 
using the constant priors of (12). In addition, we can use the 
Stasny method (1986, 1988) to estimate the expected cell 
counts under Model 1 and Model 3 that she implicitly 
assumed. However, her estimates appear to be exactly the 
same as JG], 


Attorney-general race 


Fisher Taft Others Undecided Montgomery Cordray Undecided 


Likely to vote 112 140 23 61 197 82 SY) 
Unlikely to vote 96 108 21 73 161 65 WS 
Undecided 7 11 1 4 IS 4 0 
Mayor race Treasurer race 
Coleman Teater Espy Undecided Deters Donofrio Undecided 

Likely to vote 40 32 25 30 127 119 90 
Unlikely to vote 37 47 41 56 127 90 84 
Undecided 0 2 1 0 10 7 0 
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The top table in Table 3 shows predicted values of 
elections using only “likely to vote” for the four races and 
their standard deviations in parentheses. The standard 
deviations are close to each other and show significant 
differences between the first and second leading candidates, 
except in the race for Mayor. This table also includes the 
actual election results and shows whether or not the ML 
estimates fall into the boundary solutions. 

The bottom table shows the predictions of elections using 
both “likely to vote” and “unlikely to vote” to see what 
happens if those who responded to “unlikely to vote” 
actually voted. Comparing the two tables, we may conclude 
that the winners for Governor, Attorney-General, and the 
Treasurer’s elections remained unchanged regardless of the 
likelihood of voting, whereas the winner could have 
changed in the Mayor’s election if most of those who were 
“unlikely to vote” actually voted. 

Based on Table 3, we can classify the 7 estimates, except 
NON2,,, into two groups: NON2%*, NON2%", and 
NON2?2" to the first group and the remaining four 
estimates, NON2/*, NON25", IGl,,, and NON3,, to 


Table 3 
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the second group. As expected, since the priors 6,,, for 
NON2/" and NON25* are so defined that the estimate of 
m,,, Shrinks toward the ML under an_ ignorable 
nonresponse model, these two Bayesian estimates are very 
close to /Gl,,, and hence have little advantage over the 
IG1,,. It is also interesting to note that NON3,,, is almost 
the same as /Gl,, although their loglinear models are 
differently specified. 

There is no general criterion to evaluate whether an 
ignorable nonresponse model or a nonignorable non- 
response model is appropriate. However, as stated in Chen 
and Stasny (2003), the assumption of nonignorability for a 
nonresponse may be a reasonable assumption in the 
Buckeye State Poll study because people might be reluctant 
to express their preference for an unpopular candidate, or 
their current preferences are not firm or accurate at the time 
of the interview. In this regard, the NON2}", NON23", 
and NON3,, may not be appropriate in these particular 
case studies because they are almost the same as the /G1,,, 
of Model 1. 


Prediction of elections based on the October 98 and April 98 Buckeye State Polls (the unit is % and the numbers in parentheses 
are standard deviations) 


Governor Mayor Attorney-General Treasurer 
Fisher Taft Others Coleman Teater Espy Mongomery Cordray Deters Donofrio 
Likely to vote only used 
NON2 yy, 33.2(2.75) 42.1(3.00) 24.8 31.5(4.65) 25.3(4.23) 43.2 75.6(3.71) 24.4 57.0(3.48) 43.0 
NON2/* 40.6(3.04) 48.5(3.27) 10.9 38.1(5.14) 34.2(4.78) 27.7 72s4(3,61) Zhe 52.7(3.36) 47.3 
NON23* 40.9(3.01) 50.7(3.20) 8.40 39.9(5.04) 33.6(4.83) 26.5 T1059) 29.0 52.1(3.34) 47.9 
NON23" 35.8(2.85) 44.5(3.08) 19.7 35.6(4.87) 29.3(4.51) 35.1 63.0(3.67) 37.0 54.3(3.41) 45.7 
NON23° 36.3(2.87) 45.2(3.11) 18.6 35.9(4.91) 29.4(4.52) 34.6 63.0(3.64) 37.0 53.9(3.40) 46.1 
NON25" 38.9(2.99) 47.4(3.20) 13.7 37.7(4.99) 33.6(4.77) 28.7 66.0(3.54) 34.0 515632) on. 48:5, 
IG\ yy 40.6(3.03) 51.2(3.28) 8.20 40.8(5.16) 33.4(4.76) 25.8 70.9(3.59) 29.1 $1.83:32)5 948.2 
NON3 yg 40.6(3.03) 51.2(3.28) 8.20 40.9(5.16) 33.3(4.75) 25.8 70.9(3.58) 29)1 5f.7(3:32) 48.3 
Actual result 45 50 5 39 cy 24 63 37 57 43 
Boundary yes yes yes no 
Likely to vote + Unlikely to vote 

NON2 yg 32.7(1.83) 39.4(1.91) 27.8 24.8(2.45) 26.2(2.49) 49.0 77.0(1.64) 23.0 60.2(1..93) spare 3.918 
NON2PE 41.3(1.93) 46.4(1.96) 12.3 30.7(2.68) SPACES" 32:2 72.8(1.74) DT 2 56.0(1.96) 44.0 
NON23* — 41.9(1.93) 49.2(1.95) 8.90 32.7(2.63) 36.5(2.76) 30.8 71.4(1.77) 28.0. 3 SSL. 90) aa 
NON23’ ~~ - 35.4(1.87) 41.8(1.93) 22.7 27.8(2.55) 30.5(2.62) 41.7 61.0(1.72) 39.0 57.6(1.95) 42.4 
NON2;° 36.0(1.88) 42.6(1.93) 21.4 28.7(2.57) 30.6(2.62) 40.7 60.9(1.75) B97) S7:2(1,95).2242:8 
Non2 39.11.91) 45.1(1.95) 15.8 30.7(2.63) 35.8(2.74) 33.5 64.8(1.88) 35.2 54.8(1.96) 45.2 
IG\ yg. 41.5(1.96) 49.8(1.96) 8.70 33.9(2.79)"36.1(2.74) "29.9 71.2(1.78) 28.8 55.0(1.96) 45.0 
NON3 xg 41.5(1.96) 49.8(1.96) 8.70 34.1(2.71) 36.0(2.74) 29.9 71.1(1.78) 28.9 45.0 
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Compared to actual election results, NON2,,, gives the 
worst prediction for Governor, Mayor, and Attomey- 
General because the NON2,,, lies on a boundary solution; 
whereas it provides the best prediction for Treasurer 
because it does not lie on a boundary solution. In the 
Attorney-General’s election, NON2?” and NON2}" not 
only predicted the exact actual result but also are quite 
different from the other estimates. Since NON2}" and 
NON2%" have the priors to reflect different response 
patterns between respondents and the undecided, we can 
infer that the undecided voters in the Attorney-General race 
have quite different preference for the candidate from the 
respondents (i.e., NON2%* and NON2}” allocate 19.4 % 
of the undecided voters who are likely to vote for 
Montgomery and 80.6% for Cordray, whereas the data in 
Table 2 indicates the percentage of Montgomery vs 
Cordray is 29.4% vs 70.6% among respondents who are 
likely to vote). 

To see this difference between the respondents and 
undecided voters in terms of parameter estimates and to 
examine the effect of occurrence of the boundary solution 
on the estimates under the nonignorable Model 2, we 
present the ML estimates and NON2?* estimates and their 
corresponding standard deviations for the Attorney-General 
race in Table 4. Because of a boundary solution, all of the 
ML estimates have too large standard deviations as 
expected. On the other hand, NON 2?" is very stable. Since 

Vy. = 0.0472 is the smallest and its standard deviation is 
relatively large, we ngelect Se xx, 0 avoid complexity of 
interpretation. Under By , = 0, it is not difficult to show 
that, using the estimates of “NON Bes * in Table 4, 
= 28y, + By 


log Wis 


a) = 0.09 


2/ll 


and 


on - 26), - 


2j2I 


XR) = 1.3916 


Table 4 


for each fixed j and /, and 


log ate =2(B Be 


i2k) 


,) = 0.8982 


and 


Mar = — 
log = 2B, - Biy,,,) = -1.4942 


2k2 


for each fixed i and k. Thus, by 


‘log = 26%, + Bir) = 0.09, 


2jll 


those who are likely to vote (i.e., i = 1) are 1.09 times (i.e., 


°°) more than those who are unlikely to vote (i.e., i = 2) 
among respondents (k = 1 ), whereas, by 
DUP SAL Sede rs 
log Moya 2B x, xR) 1.3916, 
likely voters of i = 1 are 4.02 times (i.e., e'*?'® ) more than 


unlikely voters of i = 2 among undecided (k = 2 ); by 


mM. 
log = 2B, + By, 


i2k1 


) = 0.8982, 


those who vote for Montgomery are 2.46 times more than 
those who vote for Cordray among respondents; whereas, 
by 
log #2 = 278 — pl, ) = -1.4942, 
*2 ca 


Ming 


unlikely voters are 4.46 times more than likely voters 
among the undecided. This implies that the response pattern 
is much different between respondents and the undecided. 


ML and the third type Bayesian Estimates under nonignorable Model 2 for Attorney-General (the standard deviations 


are in parentheses) 


1 1 1 
pe’ BeBe Be, 

-1.9487 3.2134 4.8496 
NON2y, —--3.3735 

(3.120) (8.515) (3.996) 

¥ 0.3704 -0.1490 3.3024 
NON2! 0.6860 

(0.118) (0.052) (2.501) 
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Br, XR XR, Bx, x x, Br 1” 
4.8186 2.0283 -2.7594 -0.0452 -1.5588 
(8.871) (3.120) (8.512) (0.045) (2.501) 
2.2942  -0.3254 0.5981 0.0472 -1.5450 
(2.901) (O.417) (0.052) (0.041) ~~ (2501) 
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The extent of this difference can be measured by the 
most important terms, By, and p' XR? in the nonigno- 
rable nonresponse model, Model 2. Since 


nw | My 34)/My4) _ 
=-lo = —0.3254 
faXy IVA Ean My >) 
and 
ed My iyy/My p11) _ = 0.5981, 
Ba re M12 Bre 


is the log-odds ratio that shows the log difference between 
the ratio of the number of those “likely to vote” to that of 
those “unlikely to vote” among the decided voters for 
Montgomery and the same ratio among the undecided 
voters who prefer Montgomery but who do not express their 
likelihood of voting. Whereas, B'y XR, is the log-odds ratio 
that shows the log difference between the ratio of the 
number of voters for Montgomery to the voters for Cordray 
among the decided who are likely to vote and the same ratio 
among the undecided voters who are likely to vote but who 
do not express their candidate preference. Thus, among 
voters for Montgomery, the possibility for the undecided 
voters to vote relative to not voting is about 3.67 times 


Pel ATLL 4x-0.3254 =} 
lie, ee 867 


My 1y)/My19} 


larger than the possibility of the decided, implying that 
Montgomery needs a strategy to raise the turnout of voters. 
On the other hand, among those likely to vote, the 
supporting rate of the decided for Montgomery is about 
10.94 times 


BES nail 
ie, ee ey (Es 10.94 


M, 117/91 


larger than the undecided voters for Montgomery, implying 
that most of the undecided voters not exposing their 
preference of candidate are likely to vote for Cordray as the 
Attorney-General. This also confirms the popular account 
that voters are inclined to remain “undecided” in a poll if 
they support the candidate who is seen ’as inferior in a race 
and that the voters are inclined to abstain from voting if they 
support the candidate who certainly dominates the race. 


4. Simulation study 


We consider a 2 x 2 contingency table with supplemen- 
tal margins to compare the performance of the five Bayesian 
estimates described in Section 2 for different missing 
percentages and different response patterns under the 
following nonignorable nonresponse model (i.e., Model 2): 
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log(my,) = Bo + By. + By, + Br, “" Br, 


f i 7 kl 
nS Bar, af By, s F Brix, + Brak, 


Thus, we only compare NON2,, and NON2?" for 
i = 1,...,5 im this simulation study. 

Since there are two levels in all of X,, X,, R,, and R,, 
there are 8 parameters to be determined for the simulation 
study. From the equations of 


W m,,,,/m 7 m,,,,/m 
48 = Jogi and 48 v wn) 
XyRy m,,5,/m X,R, M,,15/M1>° 
1121/7912) 112/212 
eee Ass 
XR, By, 


means that there is no difference in the ua pattern 
between respondents and undecided. The bigger B'y xR and 
vp. are, the more different the response pattern between 
regione and undecided voters is. We vary these two 
parameters from 0.2 to 0.8 with an increment of 0.2. We set 
the missias percentage to 20% and 30% by adjusting x, 


and B} R, and fixing 


My 11/91 aus My 144/117 = 
>) 3 
My, 1,/Myy1 ™M, 119/199 
and 
N= Dt ~ 1,000: 


This implies that the size and missing percentage for the cell 
of X, =1 and X, =1 are approximately 5 times and 2 
times the size of the other three cells, respectively. 

We generate a large number of samples {y,,,, i, j, k, 
/=1,2} from the above setting until we have 1,000 
random samples with boundary solutions and the other 
1,000 with no boundary solutions. The occurrence of a 
boundary solution is determined by the criterion given in 
Michiels and Molenberghs (1997) (also see Clarke (2002), 
Smith etal. (1999) for more details). Using {¥,1), Vis» 
Ysjr Vism 1, J, =1, 2} obtained from the generated 
data, the expected cell counts m,,, ’s are estimated by each 
of the five Bayesian estimates and the ML estimate 
described in Section 2. 

We calculate mean squared errors (MSEs) and absolute 
biases of NON2,,, NON2;", ..., NON23” for {Cu myys 
i, 7 = 1,2}. Then we take the mean over the four MSEs 
and the four absolute biases, which we obtain from each 
estimate to see the overall performance of the estimate. 
Similarly, we calculate mean MSEs and mean absolute 
biases for {mj + My, + Myr, i,j = 1,2} to see the 
performance of each estimate in imputing the nonresponses. 
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Table 5 shows the ratios of the mean MSEs and mean 
absolute biases of the five Bayesian estimates (ie., 
NON2}", ..., NON22"), relative to the ML estimate (i.e., 
NON2,,,) when the boundary solutions occur; whereas 
Table 5 shows the ratios when no boundary occurs. Thus, 
values less than 1 imply that the corresponding Bayesian 
estimate has a smaller mean MSE or a smaller mean 
absolute bias than the ML estimate. Both tables only show 
the cases for By wis Bry, z, and for 20% of the missing 
percentage because the MSEs and biases are almost 
symmetric about the coordinate of (By, Rp By, R,): They 
increase aS we increase the missing percentage to 30% 
while keeping the same patterns of the MSEs and biases as 
those of the missing 20%. 


Table 5 
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Table 5, where a boundary solution occurs, shows that 
NON2?*, NON23*, NON2}" have smaller MSEs than 
the ML estimate (i.e., NON2,,) for all values of By e, 
and By ,, except (BY ,, By 2.) = (0.8, 0.8). Here, 
NON23" has a smaller MSE than the ML estimate. This is 
true for the absolute biases. On the other hand, Table 6, 
where no boundary solution occurs, shows that only 
NON2%* is comparable to the ML estimate in the MSE 
although it is slightly biased. In particular, NON2$" has a 
smaller MSE than the ML estimate as long as By, R, # 0.8 
or By, 0% 0.8 (ie, The response pattern between 


respondents and nonrespondents is not very different.). 


Ratios of mean MSEs and mean absolute biases of Bayesian estimates relative to the ML estimate when boundary solutions occur 


under a 20% missing percentage (the ratios for absolute biases are in parentheses) 


(BY,n, Bx,e,) 

(0.2, 0.2) 

(0.2, 0.4) 

(0.2, 0.6) 

(0.2, 0.8) 
Far (0.4, 0.4) 
(0.4, 0.6) 
(0.4, 0.8) 
(0.6, 0.6) 
(0.6, 0.8) 
(0.8, 0.8) 


{mi + Miya + Myr) + Mizz, i, j = 1, 2} 


(0.2, 0.2) 
(0.2, 0.4) 
(0.2, 0.6) 
(0.2, 0.8) 
For (0.4, 0.4) 
Wie Myo * My te lees (0.4, 0.6) 
(0.4, 0.8) 
(0.6, 0.6) 
(0.6, 0.8) 


(0.8, 0.8) 
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NON2%* 


0.68(0.66) 
0.68(0.48) 
0.67(0.23) 
0.77(0.26) 
0.65(0.32) 
0.58(0.14) 
0.75(0.36) 
0.66(0.22) 
0.85(0.87) 
1.12(1.93) 


0.57(0.63) 
0.54(0.46) 
0.51(0.19) 
0.63(0.35) 
0.49(0.35) 
0.48(0.17) 
0.62(0.44) 
0.55(0.42) 
0.78(0.92) 
1.13(1.96) 


NON2;" 


0.47(0.22) 
0.57(0.20) 
0.73(0.66) 
1.08(1.55) 
0.69(0.57) 
0.83(0.90) 
1.46(2.07) 
1.35(1.73) 
2.27(3.19) 
3.58(5.49) 


0.27(0.13) 
0.37(0.34) 
0.69(0.94) 
1.39(2.08) 
0.54(0.64) 
0.98(1.24) 
1.81(2.33) 
1.70(1.90) 
2.91(3.43) 
4.63(5.72) 


NON2%" 


0.76(0.76) 
0.77(0.68) 
0.77(0.57) 
0.83(0.43) 
0.76(0.63) 
0.71(0.56) 
0.78(0.36) 
0.73(0.43) 
0.76(0.17) 
0.83(0.24) 


0.69(0.74) 
0.68(0.68) 
0.65(0.55) 
0.71(0.34) 
0.65(0.64) 
0.62(0.51) 
0.67(0.35) 
0.63(0.41) 
0.69(0.14) 
0.75(0.33) 


NON2* 


0.65(0.48) 
0.60(0.29) 
0.64(0.10) 
0.76(0.28) 
0.61(0.17) 
0.56(0.06) 
0.74(0.42) 
0.66(0.16) 
0.83(0.81) 
1.04(1.67) 


0.41(0.40) 
0.42(0.24) 
0.47(0.10) 
0.62(0.47) 
0.42(0.17) 
0.45(0.17) 
0.61(0.55) 
0.54(0.40) 
0.75(0.92) 
1.02(1.77) 


NON22" 


0.42(0.05) 
0.56(0.30) 
0.69(0.64) 
0.95(1.34) 
0.65(0.52) 
0.69(0.71) 
1.12(1.61) 
1.01(1.29) 
1.52(2.35) 
2.18(3.95) 


0.28(0.31) 
0.44(0.57) 
0.69(0.88) 
1.11(1.52) 
0.57(0.76) 
0.85(1.04) 
1.35(1.81) 
1.28(1.51) 
1.96(2.64) 
2.86(4.24) 
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Table 6 


Ratios of mean MSEs and mean absolute biases of Bayesian estimates relative to the ML estimate when no boundary solution 


occurs under a 20% missing percentage (the ratios for absolute biases are in parentheses) 


(Bx,e,»Bx,r,) NON2%®  NON22®  NON22® ~=NON22® — NON22* 

(0.2,0.2)  0.99(3.37) 1.05(7.00) 0.94(2.51) 0.93(4.89) _1.06(8.96) 

(0.2,0.4)  0.98(2.57)_1.21(5.13)  0.97(1.89) 1.00.26) 1.24(5.56) 

(0.2,0.6) — 1.04(2.18) 1.52(3.84) 0.95(1.67)  1.06(2.38) 1.43(3.71) 

(0.2,0.8) —-1.22(2.04)1.75(3.53) 1.00(1.48)1.13(2.14)_1.52(3.21) 

For (0.4,0.4) —1.03(2.40)_1.49(4.66)  0.97(1.69) 1.05(2.74) 1.39(4.46) 
(mig + Ming + Mp1 + Mino, i, f= 1,2} (04,06) —1.20(2.17) 2.11.85) 1.00(1.52) 1.22(2.24) _1.78(3.42) 
(0.4,0.8)  1.28(2.09) 2.36(3.67)_ 1.05(1.45)1.26(2.09) _1.86(3.12) 

(0.6,0.6) —1.22(2.16) 2.49(3.90) 0.96(1.48) 1.21(2.15) 1.90(3.32) 

(0.6,0.8) —1.52(1.99) 3.19(3.39) 1.11(1.38)1.45(1.91) 2.29(2.77) 

(0.8,0.8)  1.66(1.96) 3.64(3.27)_1.14(1.36) _1.52(1.83) 2.43(2.59) 

(0.2,0.2)  0.88(2.59)  0.89(5.66) 0.87(2.26) 0.89(4.55) __1.21(8.69) 

(0.2,0.4)  0.93(2.40)_1.27(4.86) 0.93(1.78) 1.00(3.08) _1.50(5.29) 

(0.2,0.6) —:1.09(2.11) 1.93(3.97) 0.98(1.40)_1.15(2.29) _1.85(3.61) 

For (0.2,0.8) —1.24(2.13) 2.36(3.90)1.02(1.48) 1.27(2.18)  2.06(3.19) 
{mya + myn, + Myyy, i, 7 = 1, 2} (0.4,0.4) —1.03(2.18)  1.81(4.30) 0.96(1.60)  1.12(2.62) _1.85(4.39) 
(0.4,0.6) —-1.23(2.28) 2.62(4.28) 0.99(1.48) 1.29(2.42) _2.28(3.80) 

(0.4,0.8)  1.42(2.05) 3.26(3.70) 1.07(1.42) 1.44(2.07) _ 2.53(3.09) 

(0.6,0.6)  1.33(2.07) 3.22(3.95) 0.99(1.36) 1.36(2.14)_2.54(3.43) 

(0.6,0.8)  1.65(2.09)  4.14(3.74) 1.13(1.43)1.61(2.07) 2.98(3.13) 

(0.8, 0.8) 1.91(2.02) _4.48(3.50) _1.16(1.39) _1.66(1.93) _3.03(2.83) 
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Park and Brown (1994) used NON2%* to estimate 
expected cell counts in an incomplete one-way table under a 
nonignorable nonresponse mechanism. They showed by 
simulation studies that NON2?" has a smaller MSE than 
the ML estimate although it is biased more than the ML. 
However, larger values than 1 for NON2}" in Table 5 and 
Table 6 indicate that this is not true in an incomplete two- 
way table regardless of the boundary solution and that 
Bayesian methods are not always better than the ML even 
when a boundary solution occurs. A reason that our 
simulation results differ from those of Park and Brown 
(1994) when a boundary solution occurs is attributed to the 
choice of By, Rp Bly z.) where Park and Brown performed 
their simulation only for By p = By 2. = 0.34. As shown 
in Table 5, NON25* is better than the ML when 

‘x, < 0.4 and BY, < 0.4, whereas NON22* is worse 
Ls pp 
than the ML when the response pattem between respondents 
and nonrespondents is much different (i.e., By. R, 2 0.6. or 
XR > 0.6). 

Table 7 provides the mean of the standard deviations and 
the 95% coverage probabilities for B'y , . Here, we used the 
variance formula given in (9) to calculate the standard 


deviations and the 95% coverage probabilities are the 
coverage rates for nominal 95% confidence intervals. When 
a boundary solution occurs, although the coverage 
probability of the ML estimate is closest to the 95% nominal 
coverage level, the ML estimate has too large a standard 
deviation to use in practice. Such large standard deviations 
are due to the boundary problem of the ML estimate. The 
coverage probabilities of NON25" are the closest to the 
95% nominal coverage level among the Bayesian estimates, 
while those of the other Bayesian estimates are generally 
smaller than the 95% nominal coverage level. This implies 
that the Bayesian estimates other than NON2}" under- 
estimate their standard deviations. 

When no boundary solution occurs (the second table in 
Table 7), the standard deviations of the ML estimate are 
much more stable, compared to those for the boundary 
solution case. The coverage probability decreases as By, z 
and By, R, increase. In particular, the: coverage probabilities 
of NON?*, NON}", and NON? are seriously smaller 
than the 95% nominal coverage level when the response 
pattern between the respondents and undecided voters is 
much different (i.e., By x, >0.6 and Bye, > 0.6). 


Statistics Canada, Catalogue No. 12-001-X 


50 


Choi, Choi and Park: Bayesian methods for an incomplete two-way contingency table with application 


NON2%* 


0.064(0.978) 
0.096(0.963) 
0.057(0.941) 
0.061(0.935) 
0.061(0.913) 
0.059(0.883) 


0.059(0.849) - 


0.060(0.781) 
0.057(0.657) 
0.056(0.330) 


0.056(0.959) 
0.056(0.970) 
0.055(0.945) 
0.059(0.941) 
0.055(0.828) 
0.055(0.806) 
0.056(0.771) 
0.055(0.470) 
0.054(0.402) 


NON2® 


0.093(0.973) 
0.079(0.958) 
0.071(0.941) 
0.076(0.928) 
0.096(0.956) 
0.086(0.951) 
0.084(0.926) 
0.101(0.943) 
0.089(0.913) 
0.093(0.900) 


0.062(0.937) 
0.061(0.935) 
0.059(0.959) 
0.065(0.926) 
0.064(0.899) 
0.064(0.922) 
0.064(0.873) 
0.067(0.853) 
0.065(0.817) 


ee fe standard deviations and 95% coverage probabilities (in parentheses) for Bx, R, 
(BX.r» Bx,e,)  NON2q NON2* 

boundary (1 die 89.5(0.974) 0.082(0.978) 
(0.2, 0.4) 158.3(0.959) 0.072(0.963) 

(0.2, 0.6) 135.3(0.940) 0.065(0.941) 

(0.2, 0.8) 57.4(0.930) 0.070(0.938) 

(0.4, 0.4) 153.4(0.961) 0.079(0.920) 

(0.4, 0.6) 82.2(0.955) 0.072(0.893) 

(0.4, 0.8) 51.2(0.933) 0.071(0.862) 

(0.6, 0.6) 175.5(0.946) 0.077(0.820) 

(0.6, 0.8) 159.6(0.924) 0.071(0.728) 

(0.8, 0.8) 72.8(0.920) 0.070(0.572) 

no-boundary (0.2, 0.2) 0.068(0.949) 0.060(0.959) 
(0.2, 0.4) 0.066(0.960) 0.060(0.970) 

(0.2, 0.6) 0.064(0.940) 0.058(0.945) 

(0.2, 0.8) 0.069(0.933) 0.063(0.944) 

(0.4, 0.4) 0.074(0.910) 0.061(0.836) 

(0.4, 0.6) 0.074(0.915) 0.060(0.815) 

(0.4, 0.8) 0.073(0.891) 0.061(0.786) 

(0.6, 0.6) 0.078(0.859) 0.061(0.567) 

(0.6, 0.8) 0.076(0.843) 0.060(0.515) 

(0.8, 0.8) 0.080(0.755) 0.059(0.110) 


0.053(0.017) 


0.065(0.728) 


NON2%° 


0.071(0.972) 
0.066(0.958) 
0.062(0.939) 
0.066(0.928) 
0.072(0.949) 
0.069(0.940) 
0.068(0.917) 
0.074(0.921) 
0.069(0.880) 
0.070(0.842) 


0.058(0.935) 
0.058(0.931) 
0.057(0.919) 
0.062(0.925) 
0.059(0.884) 
0.059(0.879) 
0.060(0.852) 
0.061(0.795) 
0.060(0.767) 
0.059(0.607) 


NON22 


0.060(0.957) 
0.058(0.940) 
0.056(0.922) 
0.060(0.908) 
0.060(0.911) 
0.058(0.874) 
0.059(0.846) 
0.061(0.823) 
0.058(0.737) 
0.058(0.607) 


0.055(0.922) 
0.055(0.951) 
0.054(0.909) 
0.058(0.920) 
0.055(0.824) 
0.055(0.792) 
0.056(0.763) 
0.056(0.572) 
0.055(0.556) 
0.055(0.158) 


5. Concluding remarks 


We investigated the Bayesian analysis for incomplete 
two-way contingency tables with nonignorable non- 
response. In this situation, the ML estimates often fall on the 
boundary solution. These boundary solutions can yield 
G* > 0 even for a saturated model (Baker et al. 1992; Park 
and Brown 1994). This means that the G? may not be 
appropriate as a statistic for model specification. To avoid 
the boundary solution problem and to obtain a statistic such 
as a Bayes factor for model specification regardless of a 
boundary solution, we proposed Bayesian estimation 
methods using five different priors. Two of them are new 
and the remaining three have been previously used for 
analyzing an incomplete one-way table. These two new 
priors accommodate different response patterns between 
respondents and nonrespondents. 

Data analysis shows that these new two priors are more 
reasonable in the sense that they accommodate the 
nonignorable nonresponse mechanism better and produce 
estimates close to the actual results. Moreover, with the 
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previous three priors, our simulation study shows that the 
Bayesian estimates can have larger MSEs than those of the 
ML estimates for a contingency table with no boundary 
solution and a boundary solution as well, contrary to the 
previous studies. However, when a boundary solution 
occurs, the two new priors perform better than the previous 
three priors and the ML estimates in the sense that they have 
generally smaller MSEs, smaller biases, and coverage 
probabilities closer to the nominal coverage level. 

We have briefly discussed the weighting issues at Section 
2.2. However, these issues need much more rigorous 
discussion than we did in that section. Our discussion can be 
further extended to include not only different weights but 
also response biases and other sources of biases and 
variations. These problems can be carefully developed on an 
extended paper at a later time. 
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Hierarchical and empirical Bayes small 
domain estimation of the proportion of persons 
without health insurance for minority subpopulations 


Malay Ghosh, Dalho Kim, Karabi Sinha, Tapabrata Maiti, Myron Katzoff and Van L. Parsons ' 


Abstract 


The paper considers small domain estimation of the proportion of persons without health insurance for different minority 
groups. The small domains are cross-classified by age, sex and other demographic characteristics. Both hierarchical and 
empirical Bayes estimation methods are used. Also, second order accurate approximations of the mean squared errors of the 
empirical Bayes estimators and bias-corrected estimators of these mean squared errors are provided. The general 
methodology is illustrated with estimates of the proportion of uninsured persons for several cross-sections of the Asian 


subpopulation. 


Key Words: Asian; Bias-corrected; Mean squared error; Second order accurate. 


1. Introduction 


The main motivation behind this work was small domain 
estimation of the proportion of individuals without health 
insurance for different minority subpopulations. The small 
domains were constructed on the basis of age, sex, race and 
the region where the person belongs. The National Health 
Interview Survey (NHIS) data provide the individual level 
binary response (that is whether or not a person has health 
insurance) along with individual level covariates. The data 
can be obtained at http://www.cdc.gov/nchs/nhis.htm. The 
design of NHIS is discussed in Botman, Moore, Moniarity 
and Parsons (2000). 

In a typical year the NHIS samples dwelling units, the 
collective members of each unit being referred to as a 
household, and members with a “strong” relationship being 
referred to as a family. (Structural units are more explicitly 
defined in Chapter 5.2 in the Census document at 
www.census.gov/prod/2002pubs/tp63rv.pdf). Each year the 
NHIS data contain about 40,000 households, of which over 
98% are one-family households, and contain about 100,000 
persons. For “family-type” questions, e.g., on imsurance 
coverage, all adults at home are invited to participate in the 
interview, but proxy adult response is also allowed. 
Children require an adult proxy. 

The original survey for any given year contains data on 
more than 100,000 individuals and on over 800 variables. 
Of these individuals, we have information on the primary 
response variable, namely whether a person has health 
insurance or not. In addition, there is information on 
demographic characteristics such as age, sex, race, region, 
education, income status, medical condition, disability 
conditions (if any) and many other socio-economic factors. 


For the entire US population, the direct estimates for 
these domains, namely the weighted sample proportions, are 
fairly reliable, since the sample size for each domain is 
reasonably large. This need not be the case though when our 
analysis is targeted towards specific subpopulations, such as 
Hispanics, Asians and similar minority sectors of the 
community. 

For a targeted minority subpopulation, the sample size in 
a domain is not always very large. Hence, the direct 
estimates may not be very reliable, being accompanied with 
large standard errors and coefficients of variation. This calls 
for the use of small domain estimation techniques, where 
indirect estimates are obtained for these domains based on 
implicit or explicit models. These models help building a 
link between these domains, and thus produce typically 
estimates of greater precision by borrowing strength. 

We employ both hierarchical Bayes (HB) and empirical 
Bayes (EB) methodology to obtain small domain estimates 
and find also the associated measures of precision. The 
analysis is based on a HB analogue of the generalized linear 
mixed model (GLMM) to obtain posterior means and 
posterior standard errors of the population small domain 
proportions. The method was proposed in Ghosh, Natarajan, 
Stroud and Carlin (1998). The EB approach is based on the 
theory of optimal estimating functions. We obtain EB esti- 
mators and the corresponding approximate mean squared 
error estimators by an asymptotic method analogous to that 
of Prasad and Rao (1990) and Ghosh and Maiti (2004). 
While the procedure of Ghosh and Maiti (2004) is based on 
area-level data, the present approach uses unit level data. 
Hence, by necessity, one needs some modification of the 
procedure proposed in Ghosh and Maiti (2004) in 
developing the estimators. Also, the general methodology, 
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like that of Ghosh and Maiti is not restricted only to binary 
data. The methodology is applicable to the natural 
exponential family with quadratic variance functions. 
(Morris 1982, 1983). The development of mean squared 
errors of the estimates under the proposed model is 
somewhat simpler than that of Ghosh and Maiti (2004) for 
the binary case. Moreover, like Ghosh and Maiti (2004), our 
analysis utilizes the survey weights along with the model to 
derive the small domain estimates. Thus, our method, in 
some sense, can be regarded as design-assisted model-based 
estimation. 

Survey weights attached to individual sampling units are 
usually proportional to inverses of their selection probabili- 
ties. They are often used to produce design-unbiased estima- 
tors. The classic example is the celebrated Horvitz- 
Thompson estimator. However, while such estimators guard 
against model failure, they may result in loss of efficiency if 
the assumed model is true. For example, in a simple 
Bayesian set up, if y,|9, are independently distributed 
N(0,,1), while 6, are independently and identically distn- 
buted N(p, A), (i=1,...,”), then the Bayes estimator 
(posterior mean) of O=n' Dd, 8, is Hours 
[d-B)y,+By]=(1-B)y+Bp, where B=(1+ A)". 
This estimator has Bayes risk n™'(1— B) under the assumed 
model. On the other hand, the estimator }?.,.w.y, of 0, 
with >?,w,=1 has. Bayes risk n'(1-B)+ 
E{((-B)y+Bu-SX",w.y,}. If, however, the assumed 
model is not true, for example, 0, are independently and 
identically distributed N(u, A), (i=1,...,”), where A 
departs widely from A,, then the Bayes risk of the estimator 
(1-B)y+Bp of 0 has Bayes risk n'(1—B,)+ 
(B-B,)(v-p)’, By) =(1+ 4), which can be quite 
larger than the corresponding Bayes risk of > 7, w,y, 
depending of course on B), 1 and the w,. 

The present paper produces small domain estimates of 
the proportion of uninsured persons for the Asian popula- 
tion. The estimates and measures of precision are based both 
on the hierarchical Bayesian model as well as the EB model. 
The analysis was done for all the individual years 1997- 
2000. For brevity, the results are reported only for the year 
2000. We carried out a similar analysis for the Hispanic 
population also. In this case, the number of small domains 
was 336. Since the methodology was the same as that for 
the Asians, to save space, we have not included in this paper 
that analysis as well. 

The Asian group is formally composed of the (1) 
Chinese, (2) Filipino, (3) Asian Indian, and (4) others such 
as Koreans, Vietnamese, Japanese, Hawaiian, Samoan, 
Guamanian efc. These individuals are assigned to specific 
domains depending on their age, race, gender and the region 
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they come from. There are 3 age-groups (0-17, 18-64 and 
65+), 2 Genders, 4 Races and 4 Regions depending on the 
size of the Metropolitan Statistical Area (<499,999; 
500,000-999,999; 1,000,000-2,499,999 > 2,500,000). Thus, 
the total number of domains equals 3x2x4x4=96. 
When the individuals are distributed to their respective 
domains, it turns out that many of the domains contain only 
a few observations. Indeed, there are several domains with a 
sample of size 1, while one domain has sample size zero. 

The outline of the remaining sections is as follows. 
Section 2 addresses the selection of covariates for the 
Asians. Section 3 discusses the general HB methodology 
needed for obtaining the small domain estimates and the 
associated measures of precision. Section 4 discusses the 
adequacy of the proposed HB model. Section 5 discusses an 
alternative EB methodology, finds second order correct (to 
be made precise later) mean squared errors (MSE’s) of the 
proposed EB estimators, and also second order correct 
approximation of these MSE’s. Section 6 finds the small 
domain estimates and the corresponding measures of 
precision for the Asian subpopulation in 2000 using both the 
HB and the EB methodology, and these estimates are 
compared with the direct estimates. Some concluding 
remarks are made in Section 7. 


2. Selection of covariates 


As mentioned in the introduction, the number of 
covariates exceeds 800. Inclusion of all of them in the initial 
model is impractical and unnecessary. We started with what 
we deemed to be a meaningful set of 6 covariates and used a 
fully stepwise selection process (with a significance level of 
0.05) to finally come up with the best model. 

The six covariates that we considered were: (1) legal 
marital status, (2) family size, (3) education level, (4) total 
earings from the previous year, (5) total family income, 
and (6) full time working status. 

After the stepwise procedure, our final model included, 
along with the intercept term, the covariates family size, 
education level, and total family income. 

Since the SURVEYREG procedure in SAS Version 8 
fits linear regression models and produces hypothesis tests 
and estimates for survey data, we used this procedure for 
our covariate selection. Logistic regression for covariate 
selection was not available at the time when this research 
was done. It may be noted though that SURVEYREG 
acounts for clustering and unequal weighting, and produces 
standard errors that correctly account for complex survey 
designs. 
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A general one-parameter exponential family model is 
given by 


F(%;| 8,) = explé, £99, - 


j=1,...,n,,1=1,...,k. Here y, is the response of the (ia 
unit in the i small domain, while E,, the “so-called” 
overdispersion parameters are assumed to be known, and 
are taken as 1 without loss of generality. This is because one 
can otherwise work with the transformed parameters. 
C, =5,9,- The function h is a positive function which 
depends on the y,, but not on the 0,. If y,, is binary with 
success probability p,, then 0, =logit(p,). In our 
example, y,, the response of the j" individual in the i® 
small domain, is 1 or 0 depending on whether the person 
does not or does have health insurance. We are interested in 
estimation of [,,=2/,W,p,, the domain specific 
weighted averages of the population proportions. In this 
case, the direct estimator of —1,, is D7 w,y,- These direct 
estimators are usually subject to large standard errors and 
coefficients of variation. The survey weights w, are 
assumed to be known, and are normalized so_ that 
YW, =1 for all i=1,...,4. It must be admitted though 
that often in practice, the w, are only estimates, for 
example taking into account post-stratification and non- 
response. However, the actual mechanism used to generate 
these weights are unavailable to secondary users of the data, 
and we need to assume the weights to be known. Another 
important example, not specifically considered in this paper, 
is y, ~ Poisson(A,,), so that 0, =log(A,). One can use a 
Poisson model Was based on ihe domain level counts of 
uninsured people The difficulty lies in the fact that in the 
present example, we have individual level and not domain 
level covariates. Modelling the counts via domain-level 
covariates is not possible in this situation. 

In this section, we discuss how to carry out the analysis 
for the general hierarchical Bayesian model when we are 
interested in estimating pw, = E(y,|9,)=w'(0,). Since 
y"(0,,) = var(y,, |9,,), 4; iS a one-to-one function of 90, 


YO) ACs 6) G-1) 


In particular, pu, =p, in the binary case. Specific 
applications will be considered in Section 5. 
The next stage of the model is 
0, =X; DBE ae pm Lc TA (so) 


where x, are the design vectors, or equivalently the 
predictor vectors, b is the vector of regression parameters, 
and u, are the random effects. It is assumed that wu, are id 
N(0, 07). Also, let oie eh rnex, phd baer), 
and assume that X is a full rank matrix. 

Finally, it is assumed that 6 and o; are mutually 
independent, where b has the improper uniform prior on, 


95 


R’, and o2 has an inverse gamma distribution with 
parameters c/2. al2Qies Com ) oc exp(—c/ 20? (GC, (as 
ei>i0, 

Let) y=Ons= 
KO jess 8 
given by 


kon 
n(0,b,0;,|¥) <] [[ [ £0, 16,) 


j=l j=l 


<(62) ex] 5) cS 2, - Dd 


eww Megle wand * On 
Osit-talOep )’. Then the joint posterior is 


peer he 


(oleae, cx a } (3.3) 


This is a nonconjugate Bayesian analysis, and is not 
implementable analytically. Instead, we use the Markov 
chain Monte Carlo (MCMC) numerical integration 
technique. In particular, we employ the Gibbs sampler. The 
general MCMC technique is discussed in many places. A 
convenient reference is Tanner (1996, Chapter 6). 

In order to implement the Gibbs SURI we need to find 
the full conditionals of 0, b and o. . The full conditionals 
are given by 

pays 8; x;,b)" +c vat 


2 
o |0,b, wyg{ Bea 8 6 
i y 5 5 


b|0,07,y ~N((X'X)'X’y, o2(X7 XY"); 


] 
0, | b, o., ae Ivy edexo| 0, ~~ | 


Our data analysis is based on generating samples from the 
above conditionals specialized to the binary case. Gene- 
ration of samples from the conditionals of o7 and b is 
standard. This is not so for the 6,, and requires the 
Metropolis-Hastings algorithm. For a discussion of this 
algorithm, we refer once again to Tanner (1996). 

If ay ’ denotes the sampled value of H, generated from 
the r" draw, and the number of draws is R, then the Monte 
Carlo estimate of E(u, | y) is R” oe joe Similarly, the 
Monte-Carlo estimate of var(u,,|y) 1s R- Dest ee 
(Rey hehe Finally, Monte-Carlo estimate of cov(Hy, 
Hy |») is given by a hae Si) 
(Roo nia Based on these ap ata it is now 
immediate ‘ po Elt,,| ¥]= w, E(u, |y) and 
ANC See yW; V(u, bak Se <n, Wy W, Cov(u,, 
LH, | y). In contrast, the direct unbiased Boho of Lt,, is 
given by ¥,,= Liaw Vy- 
many of these domains, the sample sizes are so small that 


However, as noted earlier, for 


these unbiased estimators are subject to large standard errors 
and coefficients of variation. 
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4. Empirical Bayes estimation 

Once again, let y, denote the response of the / unit in 
the i small domain (j=1,...,,;i=1,...,k). Also, we 
assume the exponential family model for the y,, as given in 
(3.1), but it is assumed in addition that the y, has a 
probability function or a probability density function 
belonging to the natural exponential family quadratic 
variance function (NEF-QVF) class. We may recall that 
, = E(y, |9,;)=w'(0,). With the quadratic variance 
function structure, Var(y, |9,) = Q(u,,) =Vo + Yb, + 
Ny bise where v,,v, and v, are not simultaneously zero. 
Morris (1982, 1983) has characterized distributions 
belonging to the NEF-QVF family. The family consists of 
the six basic distributions, namely, (i) Bernoulli, (11) 
Poisson, (iii) normal with known variance, (iv) geometric, 
(v) exponential, (vi) hyperbolic secant, and their convolu- 
tions. In this way, binomial, negative binomial and gamma 
distributions also belong to this family. For the Bernoulli 
distribution, v, =0,v,=1 and v,=—1. For the Poisson 
distribution, v,=v,=0 and v,=1. For the normal 
distribution with known variance o”, Gi = Se yl aand 
v, =v, =0. Once again we will assume without loss of 
generality that €, =1. 

We propose in this section EB estimators of the small 
domain means. To this end, we begin with the general NEF- 
QVF family of distributions along with a conjugate prior for 
the canonical parameter of the exponential model. Together 
they constitute an overdispersed NEF-QVF family of 
distributions. Specifically, we consider the conjugate prior 
with pdf 


n(9;;) aa exp[A{m,8, er y(8, )f ICA, m,;) Gay, 


for 0,;, where m, =P (EP ech de: death 1 = lew... Kphiere 
Xx 


, 18 the design vector associated with the j'" unit in the 
i" small domain, and g is the link function. Then (Morris 
1983), 

E(u, ) = m,; var(u,,) = O(m, )/(A — v,), (4.2) 
where we assume that A > max(0,v,). Since var(y,,) is 
strictly decreasing in A, we may interpret the latter as the 
precision parameter. 

We first obtain the Bayes estimator of 1. This is given 
by (Mormis 1983) 


l r 
BG. (ai ee ae (DY: 
(ui | Yi) agi ig) 


The above can also be viewed as the best linear unbiased 
predictor (BLUP) of 1,. To see this, we calculate 
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E(y,) = E(u,) = m,; cov(y,,H,,) = var(u,) 
A+] 
= O(n, (Av); var y,) =m, ). 
i 


Hence, the BLUP of p1,, is given by 


COV(V iis Hy) 


m,,(b) + ar 


Oi; —mM,; (b)) 


] 


aie, (4.3) 


r 
Bergen rey 
Thus the Bayes estimator of fi,,=/,w,M,, is given by 
Lij-WyE (My | ¥y)- 

In practice, however, b and 4 are unknown, and need to 
be estimated from the marginals of the y,. However, 
except for the normal distribution, these marginals are fairly 
complicated, and finding MLE’s from the marginal likeli- 
hoods can become quite formidable. Instead, we find 
estimates based on some optimal unbiased estimating 
equations (Godambe and Thompson 1989) which requires 
only evaluation of the first four moments of these marginals. 
To this end, we begin with the the elementary unbiased esti- 
mating functions ig,,=y,—-m, and g),=(y;- m,,)° - 
(A+1)/(A—v,)V(m,). In order to construct the optimal 
estimating equations, let 


(83) (33 
r Ob Ob 


Also, let 


Hoi B35; 
{Leelee ree 
3ij 4ij 2ij 


5] 


where H,,, = E(y, —m,)’ is the r central moment of Vi 
based on its marginal distribution. The optimal estimating 
equations are then given by Yi, D7, DjXi'g, = 0, where 
By = (Sj 82 )’. We obtain estimates of b and A (if they 
exist) by solving these equations. The solutions of these 
equations are found by the Nelder-Meade algorithm. 
Unfortunately, the above method fails for binary data. In 
this case, v, =—1 so that var(y,,) does not depend on i. 
Indeed, the marginal beta-binary distributions of the y,, are 
unidentifiable in A. A simple way to verify this is that if 
y|p-~Bind, p), and p~Beta(Am,A(1-m)), then 
E(y) = E(p)=m, and a binary distribution is completely 
characterized by its mean. The problem does not occur for a 
Binomial(n, p) distribution with n>2 since with the same 
marginal for p, the mgf of the marginal distribution of the 
binomial y is E[(pexp(t) + 1—- p)"] which depends on i. 
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For binary y,, 0g,,/0A =0g,,/OA=0 so that the 
second element of the vector +4,” jal Dj=i g; iS Zero. 
Accordingly, the proposed estimating equations approach 
fails to estimate A. The basic data, to be considered in our 
application, is binary, and this necessitates modification of 
the proposed procedure. 

We have thus considered the optimal estimating function 
(for known 2 ) 


x 55 [(y, — m,)/(var(y, re 


j=], j=) 


ane 


since Og,,, /0b = —Om,, / 0b. It may be noted also that in this 
case var(y,,) =V(m, ie m,(1—m,). Also, with the logistic 
representation, m, BE = exp(x! b)/[l + exp(x; b)], one gets 
(Om, / 0b = —m, (1—m,,)x,,. Thee b is Pe iidicd from the 
Sisentied Berenants pod pip XijVy = pp ae yoni mri 
Denoting this estimator by 5, the EB estimator of p,, is 
given by 


AE oe r 
——m,(b 4.4 
ta Fae rA+1 j(). Ae) 
Accordingly, the EB estimator of {f,, is i. = iW; en 


The procedure described above assumes a known A. 
One can find estimates for the y,, for different choices of 
A. In this article, we have tried 2 = 0.1, 0.5 and 1, and have 
compared the estimates with the corresponding HB 
estimates. 

Next, in this section, we find the mean squared errors 
(MSE) and also the estimated MSE’s of rie assuming 
known A. We state two theorems in this section. Some 
notations are needed before stating these theorems. Let 
M =Diag(m,,..., My, s---y Mgyy--- Mpp,) and Z(b)=X'M 
(I -M)X =}, Y/.m, (= m,) x; x70 Also, 
Diin,. It is assumed that 1<n,<C for every i, so that 
n, =O,(k), where O, denotes the exact order. The two 
theorems are now given below. 


Theorem 1. Assume 2(b)=0,(k), ie, each element of 
Xx(b) is bounded below by some constant C,, and is 
bounded above by some constant C,, where 0<C, 
< C, <0. Then an approximate expression for MSE(fi®) 


correct up to O(k™ ') is given by 


let n, = 


EB 


MSE(i,) = 


jm, (b)(1 — 


eS Wi y 


m, (b)) 


ra wm, (b)(1 my OD 


at 


210) 3 w,m,(b)(1 - my(05 | (4.5) 


o7 


Theorem 2. Assume %(b)=O,(k). Then the up hs 
AEB 


approximation to MSE(ji_) holds correct up to O(k"'). 


(la am: aa m, (b)(1 — m, (b)) ~ (1 = 2m, (b))m, (B) 


tr(x'(b) K, (b)) 
=m, 6) 26) 
tr(x'(b)K,, (b)) 


+m, (b)(1— m,(b))” x; 5'(b)x, 


aoe w,m, (b)(1 mB 


naar 


<3 16) 3 w,m, (b)(1— mB | (4.6) 


The proofs of these theorems are deferred to the 
Appendix. We will apply these results in finding approxi- 
mate estimates of MSE’s of EB estimators in the next 
section. However, before that, the following point is worth 
noting. 

If one denotes the coefficient of 4/(1+ 4)? by B, (5) and 
the coefficient of 47/(1+A)* by C, (5) in Theorem 2, then 
noting that B.(b)=O(1) and C,(b)=O(k"'), for large 
k, MSE({i) is maximized at 4 = (B,(b))/(B,(b) —2C,(6)) 
which is typically very close to 1. The resulting prior with 
i replacing 4 is the data adaptive approximate least 
favorable prior. In the example to be considered, this 
estimated 2 turns out to be 1.003 which conforms the 
above observation. 


5. Small domain estimates for Asians 


We first describe how the small domains are constructed. 
Consider the 4-tuple (k,, k,,k;,k,), where k, =1,2,3 or 4 
according as the person is Chinese, Filipino, Asian Indian or 
Islanders. Next k, =1 or 2 according as the person is a male 
or a female. Then k, =1,2 or 3 according as the person 
belongs to the age-group 0-17, 18-64 or 65+. Finally, 
k, =1,2,3 or 4 according as the person belongs to a 
Metropolitan Statistical Area (MSA) of size < 499,999, 
500,000 - 999,999, 1,000,000 - 2,499,999 or = 2,500,000. 
A small domain is now numbered by the formula 
24(k, —1)+12(k, —1)+ 4(k, -1) +k, corresponding to the 
4-tuple (k,,k,,k;,k,). For example, the small domain 
consisting of Filipino females belonging to the age-group 
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18-64 and a MSA of size 500,000 — 999,999 is numbered 
42. 

The basic data consist of y,, =1 or 0 ifthe / individual 
in the i small domain does not (does) have health 
insurance; 

Ww = the sampling weight attached to the j"" unit in the 


i small domain; 


Wy = Wy iaW, so that Lia w, =1 for each i. 
x,, = the family size of the j" unit in the i” small 
domain; 
72 = _ the education level of the j" unit in the i" small 
domain; 
x; = total family income of the j unit in the i" small 
domain; 


Let p, = E(y,). For the HB analysis, we model 


0, = logit(p,,) = by + By xi, + bX jo + B3x,3 + U;, 


il i 


fg ty Lae, 90 


The direct domain estimates are given by p,,, = D/'1WjYj- 
The corresponding hierarchical Bayes estimates are given 
by! pe = DW, E(p, | y). We use MCMC as described in 
Section 2 to obtain these estimates. They are referred to in 
the table as HB. The associated posterior standard errors are 


Table 1 


referred to as se(HB). Our hyperprior considers: 
c= 0.2, 0.02, 0.002; d =0.2,0.02,0.002. The results are 
very insensitive to the choice of the hyperpriors, and are 
reported only for c=d=0.02. In addition, we have EB 
estimators for different choices of the parameter A. The 
results are reported for 4 = 0.1, 0.5 and 1. 

Table 1 provides the various estimates of uninsured 
Asian people and the associated standard errors for the 
different small domains for the year 2000. Domain 2 is 
excluded due to zero sample size. Domain 2 refers to Male 
Filipinos in the age group 0-17 belonging to MSA’s of size 
500,000 - 999,999. The measures of precision (posterior 
s.d.’s) associated with the HB estimates are denoted by 
se(HB) and are given by the formula se*(HB)= 
var (>. W, P| y)- One of the advantages of the HB or EB 
estimates is that for domains with very small sample sizes, 
often the direct estimates of the proportion of uninsured is 
zero, whereas the former provide small but non-zero 
estimates. We chose not to collapse the direct estimates for 
domains with very small sample sizes. The unit level 
covariates were quite distinct, and there was no meaningful 
way to combine them. We note also that when A = 0.5, i.e., 
the direct and synthetic estimates have 1: 2 weight ratio, the 
EB and HB estimates are very close. 


Small domain estimates of the proportions of uninsured Asians: Year 2000 


Domain 7; Direct 97-99 HB se (HB) EB EB se (EB) se (EB) 
average A=0.5 A=1 A=0.5 A=1 
1 10 0.126 0.034 0.133 0.043 0.148 0.158 0.057 0.060 
D 0 - 0.085 - - - - - - 
3 24 0.063 0.016 0.074 0.025 0.076 0.082 0.037 0.039 
4 28 0.146 0.105 0.150 0.027 0.163 0.171 0.041 0.043 
5 20 0.138 0.265 0.143 0.032 0.153 0.160 0.043 0.046 
6 17 0.112 0.124 0.120 0.032 0.134 0.144 0.019 0.021 
7 78 0.097 0.100 0.104 0.015 0.107 0.112 0.022 0.024 
8 66 0.274 0.229 0.253 0.023 0.240 0.224 0.072 0.076 
9 5 0.173 0.000 0.164 0.061 0.160 0.154 0.078 0.082 
10 6 0.000 0.000 0.033 0.051 0.082 0.123 0.070 0.074 
11 7 0.000 0.084 0.032 0.047 0.090 0.134 0054.4 e005 7. 
12 11 0.335 0.000 0.302 0.056 0.275 0.245 0.060 0.064 
13 Ii 0.134 0.061 0.134 0.045 0.130 0.128 0.103 0.110 
14 2 0.000 0.151 0.020 0.064 0.026 0.039 0.031 0.033 
15 Dif 0.000 0.104 0.023 0.023 0.035 0.052 0.032 0.034 
16 29 0.113 0.191 0.119 0.024 0.123 0.127 0.033 0.035 
iw 27 0.120 0.223 0.127 0.025 0.141 0.152 0.044 0.047 
18 14 0.000 0.106 0.024 0.030 0.041 0.062 0.019 0.021 
19 77 0.131 0.111 0.133 0.015 0.133 0.134 0.021 0.023 
20 75 0.223 0.222 0.213 0.018 0.207 0.200 0.089 0.095 
21 3 0.000 0.000 0.022 0.056 0.028 0.043 0.070 0.074 
2D 6 0.000 0.184 0.026 0.045 0.052 0.079 0.071 0.075 
Ds 8 0.000 0.022 0.037 0.050 0.108 0.162 0.063 0.067 
24 9 0.000 0.000 0.029 0.042 0.062 0.093 0.052 0.055 
25 10 0.000 0.083 0.023 0.034 0.031 0.046 0.061 0.065 
26 6 0.000 0.018 0.020 0.039 0.029 0.044 0.031 0.033 
Dy, 32 0.098 0.041 0.105 0.023 0.108 0.114 0.035 0.037 
28 23 0.000 0.092 0.024 0.025 0.037 0.055 0.032 0.034 
29 25 0.187 0.211 0.173 0.030 0.151 0.134 0.035 0.037 
30 23 0.227 0.076 0.210 0.032 0.188 0.169 0.021 0.022 
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Table 1 (continued) 
Small domain estimates of the proportions of uninsured Asians: Year 2000 


Domain 17, Direct 97-99 HB se (HB) EB EB se (EB) se (EB) 


i 
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The HB estimates of the proportion of uninsured for 
Asians vary in the 2%-35% range for the different small 
domains excluding domain 69. Admittedly, the EB and HB 
estimates for domain 69 are very adversely affected due to 
small sample size. We also report the standard errors asso- 
ciated with the HB estimates, and estimated approximate 
root mean squares accompanying the EB estimates. The 
proposed approach largely overcomes the valid criticism 
that naive EB estimates of standard errors (which ignore the 
O(k7') term) are typically underestimates. We have also 
provided a column giving the 3-year average of the direct 
estimates in 1997-1999. This is primarily to examine 
whether domains with zero direct estimates in 2000 also 
possess the same feature in other years, and also for compa- 
rison of EB and HB estimates with these estimates rather 
than the direct estimates. It turns out that with very few 
exceptions, the 1997-1999 average do not conform very 
much to the direct estimates. However, domain 69 still has 
zero direct estimate. 

Table 2 provides the summary table for the proprtion of 
uninsured for the three age groups 0-17, 18-64 and 65+ 
individually for Chinese (Asian 1), Filipino (Asian 2), Asian 
Indian (Asian 3) and other Asians (Asian 4). It turns out that 
at this higher level of aggregation, both the EB and HB 
small domain estimates are fairly close to the corresponding 
direct estimates except possibly for the age-group 65+. This 
seems to be quite satisfactory, since at this level of 
aggregation, the direct estimates often serve as benchmarks 
for comparison purpose. 


Table 2 
Proportions without health insurance coverage by age group and 
Asian group in 2000 


Direct HB EB(A=0.5) EB(A=1) 
0-17 years 
Total 0.120 0.126 0.131 0.137 
Asian 1 0.087 0.097 0.105 0.114 
Asian2 0.046 0.062 0.071 0.083 
Asian3 0.113 0.117 0.114 0.114 
Asian4 0.165 0.165 0.171 0.175 
18-64 years 
Total 0.177 0.172 0.168 0.164 
Asian 1 0.162 0.160 0.160 0.159 
Asian2 0.137 0.137 0.135 0.134 
Asian3 0.150 0.147 0.141 0.137 
Asian4 0.219 0.208 0.203 0.195 
65+ years 
Total 0.063 0.080 0.103 G:123 
Asian 1 0.083 0.097 0.123 0.143 
Asian2 0.021 0.043 0.064 0.085 
Asian3 0.119 0.126 0.136 0.145 
Asian 4 0.055 0.075 0.100 0.123 
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6. Model diagnostics and implementation of the 
hierarchical Bayesian model 


We followed Gelman and Rubin (1992) for the 
implementation and convergence diagnostics of the Gibbs 
sampler. In particular we took 5 chains each of size 1,000 
with an initial burning period of 1,000 iterations. We 
checked the potential scale reduction factors for conver- 
gence and these appeared to be very close to unity (= | at 
convergence) for each one of the parameters. A number of 
other diagnostics criteria are available in the literature, and 
are implemented via the software CODA. A partial output is 
provided in the Figure 1. The left side shows the overlap of 
the 5 parallel chains, and the nght side shows the posterior 
inference for each parameter and the deviance (-2 log 
likelihood). For details regarding the description of the 
software that we used, we refer to Appendix C of Gelman, 
Carlin, Stern and Rubin (2004). 

A Bayesian way to check the fit of a model to data is to 
draw simulated values from the posterior predictive 
distribution of replicated data and compare these samples to 
observed data. A wide departure between the generated and 
the observed data indicates lack of fit of the model. 
Following Gelman ef al. (2004), we calculated the Bayesian 
p-values for checking the goodness-of-fit of the proposed 
Bayesian models. The general rationale behind such 
calculations is as follows. Let y denote the vector of 
observed data, € the vector of unknown parameters, 
f(yv|§) the density of y given € and II(&|y), the 
posterior density of € given y. Suppose one has drawn 
samples &"),...,& from this posterior distribution using 
MCMC simulation. Simulate now R hypothetical replicates 
of the data, say y,..., y, where y,(/=1,..., R) is 
drawn from the conditional distribution of y given the 
simulated €. If the model is reasonably accurate, these 
hypothetical replicates should be similar to the observed 
data y. This is formally done by first choosing a divergence 
variable, say d(y,&) which will have an extreme value if 
the data y are in complete disagreement with the given 
model. Then a p-value is estimated by the proportion of 
cases in which the simulated divergence variable exceeds 
the realized value of the same. Thus the estimated p-value 
(usually referred to as the posterior predictive p-value) is 
equal to Ro Diet agt dis eeecgh where J is the usual 
indicator function. One way of checking the goodness of fit 
of the model is by a scatter plot of realized values 
d(y,&) against the predictive values d(y'”, &) on the 
same scale. A good fit is indicated by about half the points 
in the scatter plot falling above the 45° line, and half falling 
below. In other words, for large samples, the estimated p- 
value will not be far away from one half. Of course, one 
may also carry out a graphical analysis by using different 
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plots for different subgroups, thereby allowing visualization 
of possible local model failure which may otherwise be 
obscured in the aggregate plot. 

There are several possible choices of the divergence 
variable d. We considered a particular one in the present 
case. Noting that E(Y;, | p,) = p, = exp(9,,)/(1+ exp(9, )), 
one can consider the squared standardized residuals 
(yy - Py) )f (pr dep,” v0 wherexn p= exp(6P)/ 
(1+ exp(6" )) are He generated values of the p, from the 
/" iteration. Then the divergence variable d is 


80% interval for each chain R-hat 
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Clearly, there are other possible choices of d. Gelfand and 
Ghosh (1998) proposed a number of divergence measures, 
and studied their properties. 
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Figure 1 Bugs model at “asian_model.bug”, 5 chains, each with 1,000 iterations 
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For the hierarchical Bayesian logistic regression model, 
the estimated p-value is 0.4216 for (c,d) = (0.02, 0.02). 
The other choices of (c,d) produce similar values. The p- 
value bigger than 0.3 is usually treated as a good fit. Thus 
the proposed HB procedure seems to work well in this 
situation. 

We have also calculated the p, = var(deviance)/2 and 
the deviance information criterion DIC or the estimated 
predictive deviance. The p, can be thought of as the 
number of ‘unconstrained parameters in the model, where a 
parameter is counted as | if it is a part of the original model 
(data distribution) and is 0 if it is associated with any prior 
distribution. The DIC is estimated as 


DIC = 2D(y, 0”) — Diy, 9) 


where D(y, 6) is the deviance calulated at the estimated 
parameters and D(y, 0) is the esimated deviance using 
posterior simulation. For details, see Gelman ef al. (2004). 

For our HB analysis p, =56.75 and DIC = 2,414.41. 
Usually the p, and DIC are used as criteria of model 
fitting and to select the model with best predictive power. 
Thus, we fit also the simple logistic regression model 
(current model without any random effects) which means 
that there is no data pooling, and the estimated p, and DIC 
are 22.60 and 2,379.55 respectively. The corresponding p- 
value is 0.3848. Thus the proposed model seems to fit the 
data reasonably well 


7. Summary, future work and discussion 


Estimating the proportion of uninsured people, especially 
among the minorities, is definitely a problem of great 
importance, and is likely to affect the policy making of 
Federal and State agencies. We have just started addressing 
this very important issue, and have provided both empirical 
and hierarchical Bayesian small domain estimates for the 
Asian subpopulation cross-classified by age, sex and other 
demographic characteristics. We have also discussed the 
adequacy of our model fit via posterior predictive p-value. 
Much work remains to be done however. In particular, we 
want to extend the present findings to the analysis of 
bivariate and multivariate binary data. 

As pointed out by a reviewer, the present analysis ignores 
household clustering in the likelihood, since the original 
survey was a household survey, and very definitely, 
insurance coverage is correlated within households. How- 
ever, we have assumed only a conditionally independent 
hierarchical model given the covariates and the random 
effects. Once, we have assigned distributions to the random 
effects, and subsequently distributions to the regression 
coefficients and the variance components, dependence is 
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built automatically in the final model, both at the unit and 
domain levels. Moreover, as mentioned earlier, adequacy of 
the hierarchical Bayesian model has been tested through 
posterior predictive p-values. 

As a final comment, the research presented here is for 
illustrative purposes only. Implementation of this method 
for policy related matters would require further considera- 
tuions of the methods and adherence to institutional 
standards for official policy release. 
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Hence, 
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Finally, we calculate, 
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The first neglected term is O, (|| bas b||°). From Sarkar and 
Ghosh (1998), b-b is asrinptoncaily N(0, © '(b)), where 
X(b) is defined before Theorem |. With the gyaumption 
that £(b) = O,(k), it follows that »-'(b) = O,(k'). Thus, 
|| b- b\|=O,(k- ’?) Hence, the first neglected term is 
O, (kK ey. New we observe that 


E(x} (b- b)} = E[(b- 5)’ x, x) (b- b)] 


= tr[x,x) E(b-b)(b-b)']. (AS) 


Statistics Canada, Catalogue No. 12-001-X 


64 Ghosh, Kim, Sinha, Maiti, Katzoff and Parsons: Hierarchical and empirical Bayes small domain estimation 


In order to find El(b- b)\(b - b)'], we proceed as follows: 
Let T(b) =i Apap i(¥, —m,(b))x;, so that T(b) = 0. 

By one-step Levey nd 0= T(b) = T(b)+[VT(b)| 
(b- b)+O,(n; ), where 
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(1—m, (b)), V(T(b)) = 5(b). Hence E[(b- bb by’ I= 
> '(b)+O(n,"”). Also, in (8. 5), we have E(x; )\(b—b)* = 
r(x, ba ee (b)) + O, (n; ') Accordingly, by (A.4) and 
(A. 5), we have the approximation 
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This leads to 
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assumption, the theorem follows from (A.2) and (A.9). 


Proof of Theorem 2. We first note that b=b+ 
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to achieve the desired approximation. To this end, we follow 
Cox and Snell (1968). 
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Now, again by a two-step Taylor expansion, 
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Since 1, = O,(k), the theorem follows. 
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A nonparametric test for residual seasonality 


Tucker McElroy and Scott Holan ' 


Abstract 


Peaks in the spectrum of a stationary process are indicative of the presence of stochastic periodic phenomena, such as a 
stochastic seasonal effect. This work proposes to measure and test for the presence of such spectral peaks via assessing their 
aggregate slope and convexity. Our method is developed nonparametrically, and thus may be useful during a preliminary 
analysis of a series. The technique is also useful for detecting the presence of residual seasonality in seasonally adjusted 
data. The diagnostic is investigated through simulation and an extensive case study using data from the U.S. Census Bureau 
and the Organization for Economic Co-operation and Development (OECD). 


Key Words: Multiple testing; Nonparametric density estimation; Seasonal adjustment; Spectral density. 


1. Introduction 


The presence of a peak in the spectrum of a stationary 
process is indicative of periodic behavior, such as 
seasonality or a trading day effect. There is a widespread 
interest in the identification of such peaks in the engineering 
and econometrics literature, since a pronounced spectral 
node will exert a potent influence on the dynamics of the 
stochastic process. A peak indicates a range of frequencies 
that offer a relatively large contribution to the overall 
variance of the stochastic process. If the strength of the 
peak, assessed through its height and width relative to 
neighboring values, is sufficiently pronounced, any model 
of the dynamics that ignores the corresponding periodicities 
will be misspecified. In both engineering and econometrics, 
one may be interested in signal extraction or forecasting, 
both of which are sensitive to the presence of spectral peaks. 

By a spectral peak, we refer to a region of the spectral 
density that has greater spectral mass than its immediate 
neighbors; a more precise definition is developed below. 
Due to the applications that we have in mind, our peaks 
have finite height, and thus correspond to stochastic periodic 
effects in a stationary process. Thus, we are not principally 
concerned with the detection of fixed (deterministic) 
periodic effects, nor with nonstationary periodic phenomena 
(though we make some extensions to this case in Section 3.4 
below), as both of these correspond to a spectral peak with 
infinite height. The vast literature dealing with the detection 
of fixed effects is discussed in Priestley (1981); for our 
applications the periodic aspects of the data are not fixed, 
but instead evolve over time. 

In this paper we focus on the application to seasonal 
adjustment. Specifically, we concentrate on so-called 
seasonal peaks, which may occur at the seasonal frequencies 
(assuming a monthly sampling interval) 1/6, 27/6, 31/6, 


4n/6, 5u/6, and 62/6. The detection of seasonality and 
residual seasonality presents an important practical problem 
in federal statistics, and the spectrum is a natural tool 
towards this end. The frequency domain approach to the 
detection and analysis of seasonality enjoys wide popularity, 
because it provides a very natural way to view quasi- 
periodic behavior. In fact, seasonality is — informally 
speaking — characterized by the presence of at least one 
seasonal peak in the spectrum (Nerlove 1964). Frequency 
domain methods are now employed in X-12-ARIMA 
(Findley, Monsell, Bell, Otto and Chen 1998) and are part 
of TRAMO-SEATS (Maravall and Caporello 2004), the 
two most widely-used seasonal adjustment programs 
available to the public. Note that frequency domain methods 
can be implemented via either a parametric (i.e., model- 
based) or nonparametric approach. We develop a non- 
parametric diagnostic, which can be invoked to determine 
the efficacy of any seasonal adjustment procedure, either 
model-based or nonparametric. As noted in Findley, 
Monsell, Bell, Otto, and Chen (1998), the use of fixed 
periodic functions alone to model seasonality is typically 
inadequate for economic data (also see the discussion in 
Bell and Hillmer 1984). 

Spectral peaks at seasonal frequencies in a seasonally 
adjusted series may indicate inadequacy of the seasonal 
filters — see Soukup and Findley (1999) for a discussion. 
At a minimum, seasonal adjustment filters should remove 
nonstationary seasonality and any fixed periodic effects — 
those phenomena in the observed series that contribute a 
seasonal pole to the spectrum. However, there is a 
consensus among seasonal adjusters that it is also desirable 
to remove some aspects of the stationary seasonality as 
well — hence the explosion of effort in developing model- 
based seasonal adjustment filters (Bell and Hillmer 1984). 
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The most important prior literature on this topic is 
Soukup and Findley (1999), which proposes using an 
autoregressive spectrum to find “visually significant” peaks 
— essentially the value of the spectrum at each seasonal 
frequency (or trading day frequency) is compared to its 
nearest neighbors, and is classified as a peak if the 
discrepancy is suitably large. This method is currently 
implemented in the X-12-ARIMA program from the U.S. 
Census Bureau (2002). One limitation of this approach is 
that it has really no statistical component: the significance is 
not statistical — i.e., it is not associated with a hypothesis test 
— and the thresholds to determine “visual significance” are 
determined in an adhoc fashion. This paper provides a 
statistical significance test for peak detection, and can thus 
be used to offer supplementary statistical evidence of the 
presence of a peak. 

Another related paper is Newton and Pagano (1983), 
which develops consistent estimators for the local maxi- 
mizers of the spectrum. Our approach is slightly different, in 
that we already know the frequencies of interest (the six 
seasonal frequencies) but seek to test for the presence of a 
statistically significant peak. Viewing the true spectral 
density f as a smooth function (this can be quantified 
through sufficiently rapid decay of the autocovariance 
function), a peak is a frequency A, such that 


f(A) = 0 — f(Ay) <9, (1) 


where f and /f denote first and second derivatives. 
Clearly, the second derivative must be negative with some 
significance in order for the concept to be meaningful. Upon 
further reflection, it seems that examining the infinitesimal 
geometry of f at the single point A, is naive, since any 
small spike in the side of a monotonic function may satisfy 
(1) while being dissociated from more intuitive notions of 
what constitutes a peak. Therefore, we must have negative 
convexity in a reasonably large neighborhood of A,. This 
thinking leads to the diagnostic of this paper: aggregate 
measures of the slope and convexity of the spectral density, 
appropriately normalized. Mathematically, these will take 
the form of kemel-smoothed periodogram estimates, but 
without the bandwidth being dependent on sample size. 

In Section 2 we develop the mathematical ideas of this 
method, illustrated through two carefully chosen choices of 
kermels. Section 3 shows how statistical estimators can be 
formulated, and how statistical peak hypotheses can be 
tested. The methodology is tested in Section 4; simulations 
provide a finite sample description of the size and power of 
our test. We further demonstrate the utility of our methods 
through an extensive case study involving 130 time series 
from the U.S. Census Bureau and the Organization for 
Economic Co-operation and Development (OECD). We 
use some concepts from the multiple testing literature 
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(Hochberg 1988) to combine tests based on the individual 
frequencies together into one diagnostic. Section 5 con- 
cludes, and all theorems and proofs are left to the Appendix. 


2. Measuring the local geometry of the spectrum 


We begin by discussing the geometry of the spectral 
density (or spectrum) of the time series under consideration. 
The starting point is to consider measures of slope and 
convexity of the spectrum that are completely deterministic 
(cf. the approach of Newton and Pagano 1983); later in 
Section 3 we will consider statistical measures. In Section 
2.1 we introduce the concepts of slope and convexity 
measures. The relevancy of these measures to peak 
identification is discussed in 2.2, while 2.3 provides two 
simple kernels as explicit examples. 

Suppose that, after suitable transformations and 
differencing if necessary, X,, X,, .... X, iS a sample from 
a zero-mean stationary stochastic process. We will use the 
notation X = (X,, X,, ...,X,,)’. The spectral density 
f(A) is well-defined so long as the autocovariance function 
y,(A) is absolutely summable, and is given by 


fa) = Dye (2) 


with i = J-1 and A e [-n, x]. It follows that the 
inverse Fourier transform yields 


1h) = = [" Qed, (3) 


a relation that we will use repeatedly in the sequel. Of 
course this relationship between y, and g holds for any 
integrable function g, not just a spectral density. Further- 
more, denoting the Toeplitz matrix associated with y, by 
X(g), it follows that 


: 1 px ey 2 
x 4 (8) = eeCs — k) = wa [ g(rAje’ KAY. 


Now from (2), f is d times continuously differentiable 
if Di--w lAl* ly, (A)| <oo. We assume that f is twice 
continuously differentiable for the remainder of the paper 
(this space of functions will be abbreviated as C’). 


2.1 Measures of slope and convexity 


The local geometry of a C’ function can be described 
through its first and second derivatives; an aggregate 
measure of these derivatives is obtained by integrating over 
a band of frequencies. Alternatively, one may integrate 
against a function A that has compact support over this 
band, so long as A provides a suitable proxy for integration 
over the band. We denote this integral via the general device 
of a functional 8 ,, where 
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0,1) = 5- f° AG Oar. (4) 


The function A will be referred to as the “kernel” of this 
functional. Hence the aggregate slope and convexity 
measures are defined by 8 ,( f) and 0 af f), where each 
dot denotes a single ee These functionals give a 
summary measure of slope and convexity of f/ over some 
band [uw — B/2, 1+ B/2] c[0, 2], and the corresponding 
kemels will therefore be denoted A, ,. We consider 
kemels with the following properties: (1) A, , is a C? 
function on [-n, 7]; (ii) A, is zero outside the band 
tp — B/2, p + B/2); (ai) oe , 18 symmetric about p on 
this band; (iv) A, (u + B/2) = 0. Condition (iii) ensures 
that the location ‘of the peak in f is not shifted by 
‘employing the kernel A, ,,. Note that we do not impose that 
‘the total integral of A, , be unity, because later we will 
employ a normalization that will automatically account for 
the total mass of the kernel. Now by (iv) and integration by 
parts in (4), we obtain 


8, = -8,°/) 
Opt (a0, (7). (5) 


These formulas are convenient, because they only require 
a knowledge of f, not its derivatives (assuming that we can 
compute Ap, , and Ag. .)- Following the extensive 
literature on kernels in nonparametric regression and 
spectral density estimation, we can start with an even kernel 
A defined on the band [-7, 7m] that satisfies (1) and 
A(+n) = 0. Then A, ,, 1s defined via 


r Use PAL pee 
A, (A) = B cn ») 


and is zero outside the band of frequencies [u — f/2, 
u + B/2]. Clearly we must impose B < 2p and 
B < 2(n — wp), so that [u — B/2, pw + B/2] c [0, a]; 
and the kernel A, , satisfies conditions (i)-(iv). Note that 
we cannot construct these types of measures for 1 equal to 
0 or 1. Using a change of variables, we see that 


Ya, ,(2) = exptthp}y,(AB/2n), . (6) 


so that the effect of 8B and p are in some sense separable. 
Note that we typically evaluate y , at non-integer values, so 
these relations are obtained by extending (3) to non-integer 
arguments. The fact that y Zc is complex-valued may seem 
troubling, but actually only its real portion will enter into 
our statistical estimators. Of course, we are ultimately 
interested in Ag , and Ag. .» Which are given by 


Ar? 21 
ee 
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and 


7 8r> ..(2n 
A, (A) = alee (A - ) 
B, py B° B 
Later, we will consider the squares of such kernels, and 
their corresponding inverse Fourier transforms. Hence 
assuming that [u — B/2, p + B/2] Cc [0, x], the squares 
are given by 


and 


Finally, we notice from (4) that we can rewrite 8 ,(/) as 
O(f) = > valh)y, (A). (7) 
h=-—00 
Thus it may be advantageous to determine the y ,(/) 
sequence from the kernel A. Taking the inverse Fourier 
Transform of the above slope and convexity kernels, we can 


construct SH Ais), E(Agi)i SAG eH) and BApawy as 
follows: 


14, 0) = = exp tin} (8/2) 
¥4, @ = re exp {ihu} y,(AB/2n), 


pa 8 ih hB/2 
rg (Obes ig? oP LH} Y 2 (AB/2n), 


2) 

Yq: (A) = —— exp {ih} yi (AB/2n). (8) 

Thus, if we have the time-domain information y ,(/) for 
the process {X,}, we can compute slope and convexity 
measures using (7) given the inverse Fourier transform 
sequence of the appropriate kernels. Since y,(h) is a 
symmetric sequence, we only need to consider the real 
portion of y ,(A) if it happens to be complex. 


2.2. Troughs and peaks 


The aggregate measures of spectral slope and convexity 
previously described provide the building blocks for 
determinants of the local spectral geometry. Our overall 
interest is in determining whether a given interval of the 
spectrum is a peak or a trough (or is monotonic). In the 
second order geometry of calculus, a local maximum has 
the defining property that the first derivative is zero and the 
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second derivative is strictly negative. Obviously this re- 
quires looking, sequentially, at a slope measure and a con- 
vexity measure, defined over the same band of frequencies. 
In order to test for the presence of a peak, the sequential 
ies can be seen as making inferential statements about 
a f) and 0, a f). Note, in making these inferential 
statements we chigeee u ahead of time, according to where 
in the spectrum we wish to detect a peak (or trough); B is 
chosen according to which frequencies we wish to exclude, 
a decision based on how local we wish our viewpoint of the 
spectrum to be. Then we say that p is a B -aggregate peak 
(with respect to A) of the spectrum if 


OAs Ow «and.» .0,C/),<,0. 


The sequential aspect comes from the idea that we gen- 
erally determine whether 6,(/) = 0 first, and then deter- 
mine the convexity; this will become more apparent when 
we consider statistical testing in Section 3.2. In a similar 
manner we define a B -aggregate trough when 9 ,( Tye 0: 
In terms of hypothesis testing for a peak, we have 


HOS On Cpyesbonh. tvs 
92 Ley Wy Oil eel Nee Ginn 


H!: 0,(f) # 0 
1 ie oN tg Bend 


The unusual aspect of this hypothesis test is that we wish 
to fail to reject H{” first, and then conditional on this test 
we want to reject H(”) in favor of the alternative H. 


2.3 Examples of kernels 


There are a host of kernels that satisfy conditions (1) 
through (iv); we can simply borrow from the literature on 
nonparametric density estimation. For example, the Parzen 
and Tukey-Hanning (TH) kernels (discussed in Priestley 
1981) are suitable, whereas the Bartlett and Daniell kernels 
are inappropriate, since (iv) does not hold. In general, one 
only needs to use (8) to determine the inverse Fourier 
transforms. In this section, we consider two examples: 
Quartic and TH. The advantage of these kernels is that they 
have easily computable first and second derivatives, and 
their inverse Fourier transforms can be obtained explicitly. 


Example |: Quartic Kernel 


We begin by considering a polynomial kernel of degree 
four, namely a quartic. Imposing all of the constraints (i) 
through (iv) yields the Wig form: 


A(X) = Oe Ith ry, 
8n* 

15 5 

A(A) = —(4n° - 4nd), and 
81° 

15 : . 

A(A) = —> (12n 4n*). 
81 
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Taking the inverse Fourier transform of the slope and 
convexity kernels (and their squares) yields 


15i( xn’ sinnth 3ncosth 3sinnth 
(OOS ee ge eer ae 4 ? 
Tt h h h 
15 (x’sinnth 3ncosth 3sinnh 
7 sie Gall weal lade Fe 
T h h h 
225( 2n‘sinnh 18> costh 
V4 (A) = niasir |! |< BLD Rowe? ROG ie 
h h 
78x’ sinth 180ncosth 18sinzh 
aS ae pen ee 
pres 225 (n*sinnth 6ncosth 
tae aida Gy so ees 
24n’* sinth S4ncosnh 54sinnh 
perplly eet MET AN lise igre w ye 


to which we apply (8) and obtain 


30i sink 3cosk 3sink 
el exp iu} (5 a eka ) 
30 sink 3cosk 3sink 
To euS eee w (2 orden re } 
L Bis 2sink 18cosk 
V2 (A) = exp {ihu [= rz 
78sink 180cosk 180sink 
Go Tig te go hi pone 
ie ae sink 6cosk 
ag (A) = exp {ih (8 _ 
24sink S4cosk 54sink 
BT eT yo 


where k=hB/2. Note that Vp (0)=240n/(7B°) and 
Y 32, (0)= 360/(B°m) follow by application of L’Hopital’s 
rule. These formulas allow us to construct the appropriate 
Toeplitz matrices for the diagnostic (as discussed in Section 
3.1 below, it suffices to consider the real part of these 
sequences). 


Example 2: TH Kernel 


A similar shape to the quartic can be obtained through 
the use of a cosine function. The following choice satisfies 
all the stated conditions on a kernel: 


ALA) ace is (1 + cosa), 
2m 

A(A) = hala ae i and 
20 

A(X) = es (—cosd). 
20 
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This function is identical to the Tukey-Hanning lag 
window, though here we apply it as a spectral window (see 
Priestley 1981). Hereafter it will be referred to as the TH 
kernel. Taking the inverse Fourier transform of the slope 
and convexity kernels (and their squares) yields 


i (sinn(h.+ 1) sinn(h - 2) 
(h)=—| ————_ -_————— |, 
iv ( h+1 h-1 
1 (sinn(h + 1) sinn(h - 2) 
a h Se fe ; 
1a) ral h+1 h-| 
1 (2sinth sinn(h+2) sinn(h-2 
Vp (yah sine) _sine=2)) ths 
16 h h+2 h-2 
v ge eres , Sinn (h+2) | sinn(h-2) . 
: 16 h h+2 h-2 
Now applying (8) yields 
i : sin(k + nm) sin(k - 72 
Ya.) = sper) (SSE See) 
TT : sin(k + m) sin(k — 1 
Yay, = — ar exp tii} peer sires seas. 
TT ¢ 2sink  sin(k + 2n 
18 Poa eset eererery 
_sin(k - 22), and 
k — 20 
2n° 2sink = sin(k + 2n 
14g) = Feexp thu) RL su 
, sink - =) | 
k — 2n 
where Bagh ie Note that Y 42 (0) = n/B° and 


ae 4n°/B° follow by application of L’Hopital’s 
arte (using the convention that sin(0)/0 = 1). These 
formulas allow us to construct the appropriate Toeplitz 
matrices for the diagnostic (again, as discussed in Section 
3.1 below, it suffices to consider the real part of these 
sequences). 


3. Statistical methodology 


Of course we do not typically have knowledge of the 
spectum /f, and thus it is usually necessary to form 
estimates from the data. In this section we describe 
statistical estimates of slope and convexity measures that are 
consistent and simple to compute in the time-domain. Under 
some mild additional assumptions, these estimates are 
asymptotically normal, which will be advantageous when 
performing hypothesis tests. In Section 3.1 the statistical 
estimates are defined, and their asymptotic properties are 
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discussed. Section 3.2 discusses the application to peak 
testing, and 3.3 gives an extension to joint peak testing, 
which facilitates an important application in seasonal 
adjustment. Section 3.4 discusses extensions to trend 
nonstationary data. 


3.1 Estimators of slope and convexity 


We begin by noting that the quadratic form (for any 
integrable function g) 
“x'E(g)X = —f" 20) 10) an, 

n Dame 
where J denotes the periodogram. Although the pe- 
nodogram is typically defined at the Fourier frequencies 
(2nj/n; 7 = 1,...,|n/2]|) we define it at a continuous 
band of frequencies as follows 


lg rs 
~ith 

=a are 

N |\t=) 


OGRE 


n-1 

y Re”, A E[-n, mr] (9) 

h=l-n 
with R(h) equal to the sample (uncentered) autocovariance 
function. This gives an elegant way of passing from the 
time-domain to the frequency-domain, and is well-known in 
the time series literature (see Taniguchi and Kakizawa 
2000). Moreover, such integrals of the periodogram are 
generally consistent, i.e., 8, (/) 0,(f) an > o, 
under mild conditions discussed below (note that the 
inconsistency of the periodogram is resolved by the spectral 
aggregation against the function g,_ as shown in the 
Appendix). Therefore, we obtain statistfeal estimates of the 
slope and convexity measures f by using a “plug in” 
approach, i.e., we simply replace f by J in 8 rae In 
particular, 


i ' i come 
Oe eh = =O) = nee X(A, ,) X, and 


oe ee ee (10) 

This definition makes use of (5), which accounts for the 
minus sign in the slope measure. In order to compute the 
estimate, we utilize the time-domain representation 
(expressed as a quadratic form). This representation is 
convenient in that we only need determine a suitable length 
of the sequences yj (h) and y-. ve (h), form the Toeplitz 
matrices D(A ) and Z(Ap, fs "and then compute the 
Capes forms. Note that the inverse Fourier transforms of 

A, , and A, _, heed only be determined once (see Section 
2.3 for some extpiieit examples) and can be done ahead of 
time, and then applied repeatedly to many different time 
series. 

In order to compute the time domain representation of 
the slope and convexity measures in (10), we utilize (8), 


— XE (A, 
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e.g., see the formulas in Examples | and 2. Of course, this 
will in general result in x(Ap ,) and ECA) being 
complex. However, even if 2(g) (where g can be 4, |, 
Ay ,, Or Ap ,,) is a complex Toeplitz matrix, X"Z(g) X 
will always be real. From (8), it is easy to see that 
X(g) = M+iN where M is real, symmetric, and 
Toeplitz, and N is real, skew-symmetric, and Toeplitz. 
Hence X' NX =0 for any vector X, so that X'X(g) X= 
X'MX. Therefore, for the purposes of computing the 
statistical slope and convexity measures, we may take the 
real part of y ,(/) in (8). 

Not only are these statistical estimates consistent, they 
are also asymptotically normal under some additional 
conditions (discussed in the Appendix). However, in order 
to construct a suitable normalization it will be necessary to 
estimate their variation. The asymptotic variance of 0, (/) 
1S 0. (f’) (if g is supported on [0, 2]), which ae be 
consistently estimated via 0.2 (I*)/2. (The factor of 2 is 
required, since the integral of /* tends to the corresponding 
integral of 2 f?—see Chiu (1988)). This can be given a 
time-domain representation as follows. Let R = 
{R( — n), ..., R(O),.... R(n — 1} be a 2n — 1 vector 
of sample autocovariances, and let X(g*) be 2n — 1 
dimensional in the following formula: R’'Z=(g?)R/2 = 
0.2 ,(I*)/2. This relationship can be easily verified using (9). 
Thus we will normalize 0 ,(/) by the square root of 
0.2 (I*)/2. Hence our nOMiona Statistical measures of 
slope and convexity are given by 


‘ aoe 9, 4) Pe OCR 
wp [0 aati? 2 n [R'E(A2,)R/2 
and 

FC) Wri ©2107 Fam BG 


Wj (J) IN ae er eee Pe ee as a ae pee 
is fo, (P)/2 n JR ¥ (Ag ,)R/2 


where the dimensions of the X matrices are either n or 
2n —1 as appropriate. The asymptotic properties of 

al ) and y. a JU ) are discussed in the Appendix. In 
ease both —Vn AY 4, WZ) and Jny-. ptt) yare 
marginally asymptotically © N(0, 1) under H® and 
H® respectively and the assumptions discussed in the 
Appendix. Simulations indicate that the variance 
normalization is slow to converge, and its correlation 
with numerator causes a degree of non-normality in 
smaller samples. Based on the histogram of the 
distribution simulated under a Gaussian white noise Null 
hypothesis with n = 360 and 10,000 replications (Figure 
1) there is close agreement to the normal distribution, 
except at the extremes in the tails. Section 4 explores this 
behavior further through simulation studies. 
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3.2 Applications to single peak testing 


We now consider the application to peak testing. Recall 
that we have an initial Null Hypothesis H$” that we must 
fail to reject in order to proceed. This can be interpreted as 
saying there is insufficient evidence to conclude that the first 
derivative (slope) of the spectral density is significantly 
different from zero. Now we know that —Jny Ay acl ) is 
asymptotically N(0, 1) under H{ and the assumptions 
discussed in the Appendix. If we further suppose that a 
sufficiently small value x is obtained for the test statistic, 
we will not be able to reject H{” with any confidence. In 
that case, we can consider the hypothesis Hj”, which we 
seek to reject; this is tested via Jny. ‘i (J). Although 
—Jny ve (J) and VJny. we (J) are asymptotically corre- 
lated (see Theorem 1 of the Appendix) we will consider 
the slope and convexity tests as if they were done separately 
(this correlation can be estimated, and used to determine the 
distribution of the convexity diagnostic conditional on the 
slope diagnostic; however, the interpretation of p -values 
becomes muddled. For simplicity, we treat the tests 
separately, one at a time, and do not explicitly account for 
the correlation). Our testing procedure is then conducted as 
follows: 


1. Perform — the 
—Vnw,, (1). 

2a Letap be the p -value associated with the first test 
statistic’s value x = —Vny. He ), with x and p 
related by p = 2®(-|x)). : 

3. If p > 0.05 (or some other pre-determined tolerance 
level) proceed; else conclude that there is no peak 


2-sided test of Hi using 


present. 
4. Perform the lower l-sided test of H(” using 
Vaya, (). 


a) Reject. “H! 2)" and pene that there is a peak if 
Vny- Pe (1) < ®'(a), where a is the level of the 
convexity test. 


3.3 Joint peak testing: Application to seasonal 
adjustment 


We now consider the situation where we wish to test for 
several spectral peaks simultaneously. Clearly we could 
design a kernel with several nodes, one at each peak, but 
this would merely be the sum of several individual spectral 
peak diagnostics. It would have the disadvantage that a 
significant spectral peak in one place could cancel a 
significant spectral trough elsewhere. Therefore, we would 
prefer a test that examines a set of spectral diagnostics 
within a multiple testing paradigm. 


Survey Methodology, June 2009 


For example, consider the context of testing for spectral 
peaks in seasonally adjusted data. There are six seasonal 
peaks of interest, but we must restrict attention to five due to 
aliasing problems (the peak at frequency m cannot be 
identified). If one or more of the spectral peaks is signif- 
icant, we must reject our seasonal adjustment procedure 
(since it has failed to remove all of the peaks); therefore, we 
are in a multiple testing situation, and will utilize a method 
that:controls the familywise error rate (FWER) proposed by 
Hochberg (1988) and described in Benjamini and Hochberg 
(page 294, 1995). Restricting attention to the issue of 
‘convexity, we have Null Hypotheses H\”) for each of the 
five seasonal frequencies. In our setting, the procedure of 
Hochberg (1988) is to compute p -values for the convexity 
test at each of the five seasonal frequencies, and order them 
AS Pay SP) S$ Pay S$ Pia) $s)» With corresponding Null 
Hypotheses denoted by H,,,. For a specified FWER of 
level a (e.g., a =0.05), let k be the largest i for which 
Pw < i(6 — i)a; then rejectall H,, fori < k. 

When using such a procedure, we should make Type I 
errors — i.e., identifying at least one seasonal frequency as 
having negative convexity when none is present — roughly 
a proportion of the time (if we were to restrict attention to 
H®, the convexity hypothesis). The advantage of the 
Hochberg familywise error rate approach (H-FWER) is that 
it dramatically improves the statistical power compared to 
other methods. The validity of this method requires 
independence of the test statistics under consideration, and 
so for this reason we take five kernels A,, ..., A; — centered 
at the seasonal frequencies 1/6, .... 5m/6 respectively — 
that have disjoint support. Then Theorem 1 can be 
generalized to obtain asymptotic independence of the five 
convexity test statistics (see the discussion after Theorem 1 
in the Appendix). Of course, we also conduct five separate 
tests of the slope at each seasonal frequency, where we must 
fail to reject in each case in order to proceed. 

As a final remark, we note that in practice a seasonal 
adjustment is rarely rejected on the basis of significant 
spectral mass at the fifth seasonal frequency of 52/6 
(Findley 2006). This is partly due to the difficulty in 
assigning an interpretation to this frequency.. Therefore, 
practitioners may be more interested in a “four-peak test” 
that focuses on the first four seasonal frequencies; one 
obtains this test by an obvious modification of the H-FWER 
procedure described above. 


3.4 Extending to nonstationary data 


The methodology given above assumes that the data are 
a sample from a stationary process. However, in the context 
of seasonal adjustment, it is usually the case that the 
seasonally adjusted data are once or twice integrated. In this 
case one would difference the seasonally adjusted data once 
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or twice, and then apply the diagnostics. Now application of 
the differencing operators 1-— B and (1 -— B)’ are 
essentially high-pass filters, which can be expected to 
attenuate residual spectral peaks close to frequency zero (in 
particular, the first seasonal frequency at 1/6). Thus it may 
be desirable to apply the diagnostic to the pseudo-spectrum 
instead; this can be done if the support of the kernel is 
bounded away from the poles in the spectral density. 
Suppose that the observed data are now Y,_,, ..., Y, for 
d the order of trend differencing (so usually d =1 or 2). 
When the observed data are differenced, we obtain the 
sample X, which is strictly stationary. The pseudo-spectral 
density of the {Y} process is g(A)=/f(A)|l-e"|7%, 
where f is the spectrum of {X,}. This pseudo-spectrum 
could be estimated via ¢(A)=J (A) |l-e "|, where J is 
the periodogram of X as before; this is the re-coloring 
approach of Nerlove (1964). Then 8 ,(g) is well-defined so 
long as A(A)|l-e |?“ is an integrable function; 
essentially we must ensure that frequency zero is excluded 
from the support of the kernel A. Since A is centered 
around seasonal frequencies in practice, we can easily 
contrive this condition. The corresponding estimator is then 


Ole On (h, 


where b(A) = |1 — e”| 74. The estimator is well-defined 
if Ab is integrable; moreover the asymptotic properties 
discussed in the Appendix for the stationary case extend to 
this case as well, so long as Ab is bounded. 

This extension may be more appealing to some 
researchers. However, the cost is that the inverse Fourier 
transform of Ab must be determined, which requires some 
additional mathematical work. In the simulation studies and 
data illustrations in Section 4 we trend difference the 
seasonally adjusted data, but do not implement the 
correction factor b in the kernel. 


4. Empirical studies 


Having developed the theoretical aspects of the spectral 
diagnostic, we now tur to its performance in practice. We 
first present some results obtained from simulation, which 
provide insight into the size and power properties of the test 
statistic in finite samples. Then we investigate the size and 
power empirically, by applying the spectral diagnostics to a 
suite of 130 time series (65 U.S. Census Bureau series and 
65 OECD series); we consider both the original and the 
seasonally adjusted series, and make comparisons to the 
Visual Significance, M7 and M8 quality control diagnostics 
of X-12-ARIMA (U.S. Census Bureau 2002). Additional 
empirical studies can be found in Evans, Holan and 
McElroy (2006). 
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4.1 Simulation study 


To evaluate the performance of our diagnostics we 
conducted several simulations. The first set of simulations 
examines size (level) for the single peak diagnostic. For this 
simulation we considered the slope and convexity 
diagnostics separately. Although in practice, when consid- 
ering the slope diagnostic, we wish to fail to reject the Null 
hypothesis H{?, here we are interested in empirically 
investigating the distributional properties, and so we impose 
the usual definition of size for this study. So we simulated 
Gaussian white noise which satisfies the assumptions of 
Theorem | as well as satisfying y,(J) = w,;U) = 0, so 
that H‘ and H\ are true. Of course, there are many 
processes for which both H{? and H\” are true simul- 
taneously — for example, any process with locally flat 
spectral density; however, due to asymptotic considerations 
it suffices to consider white noise. For a (large) sample size 
of n =360, using the TH kernel with p = 7/6 and 
8B = 2/6 (this corresponds to a kernel centered on the 
interval [0, 2/6]), 10,000 repetitions yields an empirical 
distribution of the normalized diagnostics, w,(/) and 
w;(Z), whose histograms are displayed in Figure 1. 
Henceforth, let 6 and a denote the levels associated with 
the slope and convexity tests respectively. Note that in this 
case we define level to mean the probability of rejecting 
H\(j = 1, 2) when H\ is true. Although, in practice, 
in the case of the slope hypothesis we wish to fail to reject 
we follow the strict definition of level and assume (for the 
purposes of this simulation) that the null hypothesis H{) for 
the slope holds true. Similarly the null hypothesis for the 
convexity is H®?’. Both the slope and convexity hypotheses 
are evaluated independently. Table 1 summarizes the results 
using both kernels from Section 2, for various sample sizes; 
the indicated 5, a-levels are for the nominal 5 percent 
level. Additionally, other choices of 4 and B (not shown 
here) yielded similar results. As depicted in this study, in 
smaller samples, we observed skewness in the distribution 
which seems to be due to correlation between @,(/) and 
0.7 *). Also, note that the size for the convexity test is 
larger for the quartic kernel than for the TH kernel. 

Next we consider the empirical power for our single peak 
diagnostic. In this setting we evaluate the power based on a 
joint test of the slope and convexity. Specifically, we wish 
to fail to reject H{) while simultaneously rejecting HY, at 
6 = a =0.05, and thus correctly identify spectral peaks. 
Since our composite Null hypothesis is that there is no peak, 
the Alternative hypothesis includes processes such as the 
AR (2) given by 

(1 — 2pcoswB + p°B’)X, = g, (11) 


with white noise variance o’, associated with some fixed 
frequency @ € [0, m]. The spectrum associated with the 
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process in (11) is given by f(A) = o°|1 —2pcoswe™ + 
p°e°"|~*| which is maximized at 4, = cos '(cos@(1 + 
p°)/2p). Therefore one can explore the power of a peak- 
testing procedure by simulating from (11) with various 
choices of p, w, and o. Table 2 presents the result of 10,000 
simulations, of various sample sizes, from the AR(2) cycle 
model given in (11) with peak at p = 72/6 and bandwidth 
set at B = 1/6. The peak strength is parametrized through 
p, which we vary from 0.85 to 0.95; clearly H and H” 
are both true for this model. In other words, there are spectral 
peaks, of different heights, at A = n/6. This AR cycle 
model was chosen because it provides a convenient para- 
metrization of spectral peak location and shape. Additionally, 
this choice of 8 is compatible with the seasonal adjustment 
setting, as this provides the maximum window width while 
avoiding overlapping spectral peaks. As expected, the power 
of our diagnostic increases with sample size and peakedness, 
ranging from 0.227 (quartic kemel) in small samples having 
weak spectral peak to ~ 0.95 (TH kemel) in larger samples 
having a more pronounced spectral peak (see Table 2). Note 
that in this procedure the innovation variance is set equal to 
one, but it is immaterial due to the normalization of the 
diagnostic. In summary, both the quartic and TH kemels 
possess decent size and power properties. Generally, the 
quartic kernel seems to have superior size and power, so it 
would be preferable for spectra of this form (note that the 
lower power of the TH kemel is in part due to its being under- 
sized). Additionally, it seems that smaller values of B (results 
not shown) require a greater sample size; a smaller B 
corresponds to a more refined “viewing” of the spectral peak, 
which would require more data to handle the resolution. 
Although the individual peak testing scenario provides the 
foundation for our joint testing framework, as noted, the joint 
testing framework provides important methodology for 
applications in federal statistics. The application of impor- 
tance is the evaluation of effective seasonal adjustment 
through the exploration of residual seasonality. Thus, it is of 
particular interest to know how our multiple testing approach 
performs in simulation. Therefore, in order to investigate the 
size and power associated with our joint test, we simulated 
10,000 repetitions from a Gaussian white noise process and 
from an AR(25) model obtained as a fit to the Current 
Employment Series (Employed Males, aged 16 to 19). Our 
goal in the power study was to construct an AR(p) process 
(because of its ease in simulation and desirable theoretical 
properties as a parametric spectral estimator — see Parzen 
1983) with (stationary) spectral peaks that are realistic, or 
close to what might be found in practice. Thus we obtain our 
AR(25) model — fitted via maximum likelihood using 
AIC — which has similar seasonal dynamics (local spectral 
behavior) to the Current Employment Series (CES). 
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Table 1 


Results of size simulation for the single peak diagnostic. Here p = B 
diagnostics were investigated separately for both the quartic and TH kernels 


x/6 and 10,000 repetitions were used. The slope and convexity 


a -level 


Size for Single Peak pt = 
Slope 
Quartic Kernel TH Kernel 
n Mean Stdev 6 -level Mean Stdev 5 -level 
120 0.003 0.903 0.007 0.011 0.903 0.008 
144 -0.004 0.920 0.014 -0.011 0.927 0.015 
180 -0.003 0.942 0.022 0.002 0.920 0.017 
288 0.003 0.954 0.027 -0.002 0.950 0.025 
360 0.003 0.962 0.032 -—0.009 0.954 0.031 
Table 2 


Results of power simulation for the single peak diagnostic. Here p = B 


B = w6 
Convexity 
Quartic Kernel TH Kernel 

Mean Stdev a -level Mean Stdev 

0.065 0.852 0.032 0.025 0.888 0.018 
-~0.077 0.882 0.042 0.006 0.892 0.025 
-0.071 0.892 0.043 0.005 0.902 0.028 
-0.072 0.921 0.051 -0.006 0.926 0.033 
-~0.056 0.922 0.051 0.006 0.951 0.040 


n/6 and 10,000 repetitions were used. The alternative 


hypothesis is given by the AR(2) model defined by (11). The slope and convexity diagnostics were investigated simultaneously for 
both the quartic and TH kernels using 5 = a =0.05 for both tests (see Section 4.1) 


Power for Single Peak pp = B = 1/6- 


Quartic Kernel 


n p = 0.85 p = 0.90 Oi 0.95 
120 0.227 0.438 0.758 
144 0.287 0.532 0.856 
180 0.354 0.643 0.923 
288 0.447 0.755 0.949 
360 0.601 0.872 0.937 


(8, a) = (0.05, 0.05) 


TH Kernel 
p = 0.85 p = 0.90 p = 0.95 
0.147 0.335 0.670 
0.208 0.431 0.799 
0.272 0.567 0.901 
0.372 0.706 0.950 
0.537 0.859 0.948 
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To evaluate size we considered both a test based on 
convexity alone and a test based on the slope and convexity 
simultaneously. The tests based on convexity alone (C) 
were performed at the nominal «a -levels of 0.05 and 0.10, 
using the H-FWER method to control the FWER. The tests 
based on both the slope and convexity simultaneously (S, C) 
were performed as follows: 

1. Perform multiple tests of convexity, HY’, using the 
H-FWER method to control the FWER at level a 
(which is either 0.05 or 0.10). 

2. For any peaks found significant in Step 1 perform 
individual slope test, H{?, at level 5 (which is either 
0.10 or 0.25). Note here we wish to fail to reject Hf)’ 
in order to declare any “peaks” as statistically 
significant. 

3. Declare there is a statistically significant peak if 
Step 1 finds any seasonal frequency with significant 
aggregate convexity in the spectrum, and if Step 2 
simultaneously fails to find any significant aggregate 
slope for the corresponding seasonal frequency. 


The results of this simulation are summarized in Table 3. 
One aspect of this procedure that deserves further 
explanation is Step 2 where 6 (the level for the slope test) is 
taken equal to 0.10 and 0.25. Although the slope testing 
aspect of the procedure is conducted on an individual peak 
basis, it seems reasonable to try and be conservative. The 
issue here is that even if some of the individual slope tests 
are rejected we may still proceed in other cases. Thus the 


Table 3 


situation encountered here differs from the classical “no 
peaks” hypothesis which can be rejected if a single peak is 
found. Of course, since we are conducting each slope 
hypothesis test on an individual peak basis, any 6 -level 
greater than 0.05 would be considered more conservative. 

While we cannot expect the combined procedure (S, C) 
to have size approaching the nominal (because using the 
slope test throws off the Type I error rate), neither is the size 
highly accurate in the case of using the convexity test alone 
(C), as can be seen by examining the a =0.10 case with 
n = 288, 360. Here the convexity for the quartic kernel is 
over-sized, whereas in the single peak case the convexity 
test has accurate size (Table 1) for these sample sizes. Note 
that H-FWER only produces an approximately correctly 
sized procedure; another factor is that the five peak tests are 
only asymptotically independent. It is for these reasons that 
the empirical size found in Table 3 differ somewhat from 
the nominal levels. 

To investigate power we considered the same 3 step 
procedure as outlined above. However, for this simulation 
we only considered joint slope - convexity testing and 
examined four pairs of (6, a) levels; (6, a) = 
(0.10, 0.05), (0.25, 0.05), (0.10, 0.10) and (0.25, 0.10). The 
results of this simulation (Table 4) indicate tremendous 
power even at sample sizes as small as n = 120. This is 
extremely important as n = 120 is representative of the size 
samples encountered in practice when conducting seasonal 
adjustment (7.e., 10 years of monthly data). For samples 
sizes n = 144, greater than 90% power was achieved. 


Results of size simulation for the multiple peak diagnostic. Here 10,000 repetitions were used. The convexity test was investigated 
separately with the FWER controlled at a =0.05 and a =0.10 using the H-FWER method. Additionally, the slope and convexity 
diagnostics were investigated simultaneously using the H-FWER method for the convexity controlling the FWER at a =0.05 and 
a =0.10, while the slope was evaluated at 6 =0.25 and 6 =0.10. Both the quartic and TH kernels were used for both tests (see 


Section 4.1) 


Size for Multiple Peaks H-FWER 


C=0.05 (S, C) = (0.10, 0.05) (S, C) = (0.25, 0.05) 
n Quartic TH Quartic TH Quartic TH 
120 0.006 0.002 0.006 0.002 0.005 0.002 
144 0.009 0.002 0.011 0.002 0.008 0.003 
180 0.019 0.005 0.020 0.006 0.017 0.005 
288 0.031 0.009 0.026 0.008 0.025 0.008 
360 0.042 0.012 0.045 0.019 0.035 0.015 
Table 4 


C=0.10 (S, C) = (0.10, 0.10) (S, C) = (0.25, 0.10) 
Quartic TH Quartic TH Quartic TH 
0.076 0.047 0.070 0.044 0.076 0.046 
0.087 0.053 0.090 0.051 0.086 0.050 
0.107 0.062 0.097 0.059 0.093 0.057 
0.117 0.069 0.116 0.069 0.112 0.068 
0.140 0.087 0.133 0.084 0.129 0.087 


Results of power simulation for the multiple peak diagnostic. Here 10,000 repetitions were used. The slope and convexity 
diagnostics were investigated simultaneously using the H-F WER method for the convexity controlling the FWER at a =0.05 and 
a =0.10. For the slope 6 =0.25 and § =0.10. Both the quartic and TH kernels were used for both tests (see Section 4.1) 


Power for Multiple Peaks H-FWER 


(0.10, 0.05) (0.25, 0.05) 

n Quartic TH Quartic TH 
120 0.877 0.897 0.860 0.881 
144 0.943 0.952 0.942 0.950 
180 0.989 0.992 0.989 0.992 
288 1.00 1.00 0.999 0.999 
360 1.00 1.00 0.998 0.999 
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(0.10, 0.10) (0.25, 0.10) 
Quartic TH Quartic TH 
0.997 0.997 0.996 0.998 
0.999 1.00 0.999 1.00 
1.00 1.00 0.999 1.00 
1.00 1.00 0.999 1.00 
1.00 1.00 0.998 0.999 
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4.2 Case studies 


We also considered 130 time series, 65 from the U.S. 
Census Bureau and 65 from OECD. These series consist of 
35 U.S. Manufacturing series, 10 U.S. Housing series, 10 
U.S. Import/Export series, and 10 U.S. Retail series; there 
are also 22 German series, 15 Euro-area series, 11 French 
series, and 17 Great Bnitain series from OECD, covering the 
sectors of manufacturing, retail, wholesale, foreign trade, 
unemployment, and industry. For every series, we computed 
the seasonal peak tests for both the raw data (logged) and 
the seasonally adjusted data (logged) — using the x11 
specification of X-12-ARIMA — with both the quartic and 
TH kemels. We employed the H-FWER procedure 
controlling the FWER at a =0.05, 0.10, and where the 
threshold for the slope tests was 6 = 0.25 and 6 =0.10 at 
each peak (see Section 4.1). Note that 6 =0.25 and 
6 = 0.10 produced similar results and thus, for the sake of 
brevity, only results for 6 =0.10 are presented here. The 
results for 6 =0.25 are available upon request from the 
first author. For both the raw and seasonally adjusted data, a 
single trend difference was used (as is the case for the 
Visual Significance diagnostic, described below) before 
applying the seasonal peaks test. 

In addition, we present the M7 and M6§8 statistics as well 
as the results of the Visual Significance (VS) diagnostic 
both before and after adjustment. The M7 quality control 
statistic measures the amount of stable seasonality relative 
to the moving seasonality in the original series, with values 
greater than | indicating that the seasonality in the series is 
not identifiable (Lothian and Morry 1978); similarly, the 
M8 statistic measures the size of the fluctuations in the 
seasonal component, with a similar interpretation. We also 
considered the robust nonparametric Kruskal-Wallis test 
(U.S. Census Bureau 2002) for the presence of seasonality 
assuming stability. VS is based on an AR(30) spectrum 
estimate of the raw and seasonally adjusted series, and is 
described in Soukup and Findley (1999). For Tables 5-10, 
each cell entry lists which seasonal frequencies were found 
to have a significant peak, with j corresponding to 1 //6 
fon, j-=wa ge. 4,.5;\ anpentry off indicates that no 
peaks were detected. For the M7 and M8 diagnostics, only 
the value is reported since there is no associated p -value 
(and they are only pertinent to the raw series). 

The results of this empirical study can be found in Tables 
5-10. All of the Kruskal-Wallis statistics were significant 
with p =0.000, so these are not reported in the tables. The 
set of columns corresponding to the “Onginal Data” 
heading can be seen as giving empirical power (for each 
subset of series), assuming that each series is indeed 
seasonal and has seasonal spectral peaks. That is, the Total “ 
correct ” number gives the proportion of times each method 
correctly identified seasonality, and hence this proportion 1s 
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a crude proxy for empirical power. We also report the 
average number of peaks that were identified, which is an 
empirical measure of the efficacy of the methods (the more 
peaks correctly identified, the better). The set of columns for 
“SA Data” gives an empirical size (for each subset of 
series), assuming that seasonal adjustment has indeed 
removed the spectral peaks. These are rough considerations, 
since we do not really know a priori whether the SA Data 
has been adequately adjusted. 

The VS identifies all of the raw series as seasonal and 
most of the SA series as having no spectral seasonal peaks; 
the M7 and M8 diagnostics perform similarly, though of 
course they do not indicate which seasonal peaks are present 
in the raw data. Our procedure indicates a few cases (when 
a =0.10 for the convexity tests) where the adjustment may 
be inadequate, but these are within the scope of the expected 
proportion of Type I errors. For the raw series, the empirical 
power (i.e., total proportion correct) for our method ranges 
between 0.66 and 0.89, with higher power for the a =0.10 
level, as expected. In many cases the indicated peaks are the 
same as VS, but sometimes are quite different. Note that the 
average number of peaks detected for raw series was 
typically much higher for our procedure over the VS 
method, which often had an average around 3.2. When the 
a level was increased from 0.05 to 0.10, our method 
naturally increased in the average number of peaks detected; 
VS cannot be tuned in this way. Conversely, for SA data the 
average number of peaks detected tended to be less than one 
for our method (with the exception of the German series). 

The results are fairly similar for the quartic and TH 
kernels. Although the M7, M8, and VS diagnostics have 
slightly better performance than our spectral peak procedure 
with a =0.10, it is important to note that our method 
provides a level of detail that M7 and M8 cannot replicate, 
while the VS diagnostic does not provide a p -value for any 
of the peaks (neither do M7 or M8). Overall, we find the 
results to be very encouraging and informative. 


5. Conclusion 


This paper presents an innovative approach to the 
statistical identification of spectral peaks. The convexity 
diagnostic computes an average of the periodogram 
weighted by the second derivative of a typical kernel, such 
as the Tukey-Hanning lag window. Implicitly this type of 
statistic involves a comparison of an average of the peri- 
odogram near a given frequency to its average somewhat 
further out; this follows from the general shape of Ay, re 
The slope diagnostic helps to screen out cases where there is 
negative convexity but also a large increase/decrease in the 
spectrum. That the method actually works as intended is 
borne out by the simulations and analysis results reported in 
Tables 1-10. 
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Table 5 
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Data analyses for 35 Manufacturing Series (U.S. Census Bureau) comparing our multiple peak diagnostic with VS, M7, and M8 
diagnostics. Our multiple peak diagnostic uses the H-F WER method to control the FWER at a =0.05 and a = 0.10 and examines 
the slope at 56 = 0.10 (see Section 4.2) 


Data Analyzes — Manufacturing Series 


Original Data SA Data 
H-FWER 0.10/0.05 H-FWER0.10/0.10 VS M7 M8 _  H-FWER0.10/0.05 H-FWER0.10/0.10 VS 
series quartic TH quartic TH quartic TH quartic TH 
M 2345 1235 2345 1235 12 0.24 «= 0.39 ) 6 6 6 6 
M, 12345. 9 12345 612345. 95912345. 1235020" O32 i) 6 6 B 6 
M; 12345, biel 23-45 nsortunl 2345): sirte 1345 pum 1235, 0280 O46 ) i) 6 6 6 
M, 6 6 12345 6 12 0.28 0:44 i) 6 6 6 p 
Ms 12345 12345 12345 12345. 12345. «0.27 0.47 6 6 12345 12345 6 
Mg 6g 6 123 123 120.28 »=—-0.49 6 6 6 6 6 
M, p 6 123 123 24 «0.50 (0.79 p p 6 6 6 
Mg 12345 12345 12345, 19345 193450 .1 Sea 0.37 6 i) 6 6 6 
Mg £2345 1 jh) 2345 beet 2345uh CIA 124 OAD UN Os 6 i) 6 $ 6 
Mio g pg 6g 6 I 0.38 0.72 6 6 p 6 6 
Mi, 6p 6 12345 1234 125 0155 02g 6 p 6 1) 6p 
M2 1234 1234 12345 12345 1234s :0.30—s(0.54 6p 6 p 6 i) 
Ma 6 6b 1234 1234 «1234 «0.24. 0.39 i) i) 6 6 6 
M5 12345 12345 12345 12345." © 12345... 0.23. 043 6 6 6 6 6 
Mg 1234 1234 1234 1234-1234 «0.23 «0.40 p 6p 6° 6 6 
Mi pg 6 1234 12345 12 064 0.66 p ) 6p 6p i) 
Mg 12345 12345 12345 12345 245 0.20 0.37 i) ) ) p 6 
Mio 6p fs) 6 6 4 0.86 1.00 6 6 p 6 6 
M9 p p p 12345 4 056 0.84 6p p 6 i) 6 
My 12345 12345 12345 12345 1234S 037s 0.58 6p p 6 i) 6 
My 12345 12345 12345 12345 1234s 0.26 ~—s«0.45 p ) 6 6 6 
M3 12345 12345 12345 12345), 12340" "020 10.47 6p 6 6 p i) 
M4 12345 12345 12345 12345 2345 «0.26 (0.43 6b p i) 6 6 
M)s5 12345 12345 12345 12345 12345. 0.27: 0.42 B 6 p i) 6 
M6 12345 12345 12345 129457 °5.1235.,; £037) nebo B 6 6 6 6 
M7 1345 1234 1345 1234 2345.0 (10: 25gre0.22 6 B B p 6B 
Mo 6 B p 6 24 «0.57~=—(0.44 6 6 6 6 6 
Mo 6b 12345 12345 12345 24.° 078 mets 6 6 B 6 6 
M39 123 1234 12345 12345 245 045 0.65 6 6 6 6 6 
M3, 6 6 123 123 4 0.64 0.46 6 6 1234 1234 6 
M3> 1235 12345 1235 12345 12345. -0.21_—S (0.37 6 6 6 6 p 
M33 12345 12345 12345 1234 1234 0.24 0.38 p 6 i) 6 6 
M34 12345 12345 12345 12345 234. 0.46 (0.85 6 i) 6 6 6 
M35 12345 12345 12345 12345 2345 «0.25 (0.66 6B B 6 6 6 
M36 12345 12345 12345 12345 5. acai We ae he 6 ) 6 6 ) 
Total “correct” 23/35 24/35 31/35 31/35". 4635/35 3485 33/3508 35135 35/35 33/35 33/35 35/35 
Average Number 3.09 3.29 4.09 4.09 3.23 0 0 0.26 0.26 0 
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Table 6 

Data analyses for 30 U.S Census Bureau Series (10 Housing, 10 Import/Export and 10 Retail Sales) comparing our multiple peak 
diagnostic with VS, M7, and M8 diagnostics. Our multiple peak diagnostic uses the H-FWER method to control the FWER at 
a =0.05 and a =0.10 and examines the slope at 6 = 0.10 (see Section 4.2) 


Data Analyzes — Manufacturing Series 


Original Data SA Data 
H-FWER 0.10/0.05 H-FWER 0.10/0.10 VS M7 M8 H-FWER0.10/0.05 H-FWER 0.10/0.10 VS 
series quartic TH quartic TH quartic TH quartic TH 
MW1Fam 12345 12345 12345 12345 PPO A Tt: 25 g g 1) g 6 
NWTot 12345 12345 12345 12345 12.00, 184031 6 g 1) 6 g 
NE1Fam 12345 12345 12345 12345 12) “0/164 0.33 g 6 g 6 g 
NETot 12345 12345 12345 12345 12379 '0.25847027 g g 6 gb 6 
S1Fam 12345 12345 12345 12345 125%) 0.2288"0.47 g 6 6 6 g 
STot 124 124 1245 1245 125° /0.29°*'0.57 g 6 g ) g 
US1Fam 12345 12345 12345 12345 1258 Y017 . 0.39 1) g 6 6 6 
USTot 12345 12345 12345 12345 1257 ©0205, 0.42 6 6 6 6 g 
WI1Fam 1234 1234 12345 12345 1257 70:22 710.44 6 6 6 6 g 
WTot 1234 1234 12345 1234 120 -0:27730.56 g 6 g g 1) 
Total “correct” 10/10 10/10 10/10 10/10 10/10 10/10 10/10 10/10 10/10 10/10 10/10 10/10 
Import/Export Series 
Original Data SA Data 
H-FWER 0.10/0.05 H-FWER 0.10/0.10 VS M7 M8 H-FWER0.10/0.05 H-FWER 0.10/0.10 VS 
series quartic TH quartic TH quartic TH quartic TH 
M00120 12345 6 12345 12345 125 0.23 0.48 g g 6 6 6 
M00190 12345 12345 12345 12345 1235 038 0.59 6 6 6 6b 6 
M3000 12345 12345 12345 12345 234 048 0.95 6p p 6p p p 
M3010 1234 1234 1234 12345 2345 0.52 0.88 g 6 6 6 6 
M12060 12345 12345 12345 12345 9123 ©0535 077 6 p p 6 6 
X3 12345 12345 12345 12345 2345 0.57 0.94 6 6 g 6 6 
X00300 134 134 134 134 2 056 0.97 6p p b p B 
X3020 12345 12345 12345 12345 12345 0.39 0.70 6 p 6 b p 
X3022 12345 12345 12345 12345 23 0.69 1.04 6b 6p p p 6 
X10140 1234 1234 1234 1234 15. 029 047 6 6b 6 B 6 
Total “correct” 10/10 9/10 10/10 10/10 10/10 10/10 9/10 10/10 10/10 10/10 10/10 10/10 
Retail Series 
Original Data SA Data 
H-FWER 0.10/0.05 H-FWER 0.10/0.10 VS M7 M8 H-FWER0.10/0.05 H-FWER 0.10/0.10 VS 
series quartic TH quartic TH quartic TH quartic TH 
s0b441x0 12345 12345 12345 12345 135 022 041 p p p p 6 
s0b 44000 12345 12345 12345 12345 2345 0.12 0.26 b p 6p i) 6p 
s0b 44100 12345 12345 12345 12345 135 0.21 0.40 6p 6 p 6 1) 
sO0b 44130 12345 12345 12345 12345 211935. 028. 40 g ) 6 ) 6 
s0b 44200 12345 12345 12345 12345 12345 0.13 0.27 6 6p 6 p p 
s0b 44300 1234 12345 1234 12345 12345 0.12 0.18 6 p 6p p p 
sOb 44312 1234 1234 1234 1234 12345 031 0.48 6 6 6 6 6 
s0b 44400 12345 12345 12345 12345 1235 0.16 0.32 p 6B p p p 
s0b 44410 12345 12345 12345 12345 11035 O44 032 p B p p B 
s0b 44500 12345 12345 12345 12345 235 0.14 0.23 g 6 6 6 g 
Grand Total “correct” 30/30 29/30 30/30 3030 30/30 30/30 29/30 30/30 30/30 30/30 30/30 30/30 
Average Number 4.67 45 4.77 4.8 3.23 0 0 0 0 0 
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Table 7 

Data analyses for 22 German OECD Series comparing our multiple peak diagnostic with VS, M7, and M8 diagnostics. Our 
multiple peak diagnostic uses the H-FWER method to control the FWER at a =0.05 and a =0.10 and examines the slope at 
5 = 0.10 (see Section 4.2) 


Data Analyzes -OECD DEU 


Original Data SA Data 
Series H-FWER 0.10/0.05 H-FWER0.10/0.10 VS M7 M8 — H-FWERO.10/0.05 H-FWER 0.10/0.10 vs 
DEU quartic TH quartic TH quartic TH quartic TH 
PRMNCG03 12345 12345 12345 12345 12345 0.17 0.33 6 b 6 B v7) 
PRMNCSO1 1234 1234 1234 1234 235 | 075. 079 6 6 B 6 6 
PRMNIGO1 12345 12345 12345 12345 12345 0.28 0.48 6 6p 6p 6 6b 
PRMNTOO1 134 134 134 134 12345 0.26 0.46 6 g 6 6 1) 
PRMNVGO1 1235 1235 1235 1235 2345 0.29 0.46 6 6 B p B 
SLMNCD01 1245 1235 1245 1235 23. 0.22 ~«0.40 6 6 p ) 6 
SLMNCNO1 1345 2345 1345 2345 123" 0376? 0.72 6 6 B B 6 
SLMNDMO01 2345 2345 2345 2345 123. 032 0.63 6 6b B B 6 
SLMNEX01 12345 12345 12345 12345 2. “0327-651 1:5 6 12345 12345 gb 
SLMNIGO1 2345 2345 2345 2345 i273. “Oo 665 ) g 6 ) g 
SLMNTOO1 245 345 245 345 23 0.20 0.66 6 B 234 6 6 
SLRTCRO1 1345 1345 1345 1345 1234 0.19 0.51 p 6 p B p 
SLRTTOO1 12345 12345 12345 12345 12345 0.12 0.25 6 6 6 B p 
SLRTTOO2 12345 12345 12345 12345 12345 0.13 0.29 6 6 6p 6 6 
SLWHTO01 2345 2345 2345 2345 123. 0.20 0.62 6b 6 134 123 6 
SLWHTO02 2345 2345 2345 2345 123 0.20 0.62 ) 6 134 123 6 
UNLVRGO1 23 23 23 23 124 0.23 0.48 g g 1 g 5 
UNLVSUMA 345 345 345 345 12 030 0.53 12 pb 1245 12345 6 
UNLVSUTT 234 234 234 234 Iss 024% 0.53 pb pg 45 45 25 
UNRTRGOI 235 1245 235 1245 123. 0.19 0.59 ) 6b 2345 2345 6b 
XTEXVAO1 1234 1234 1234 1234 23° O28" 097 6 6 6p 6 6 
XTIMVAOI 234 1234 2345 12345 237 OST 0.95 pb ) 6 p 6p 
Totaleorrect’ 22/22 22/22 22/22 22/90) 92/2 22/22. 22/7228 "'20/22 22/22 14/22 16/22 20/22 
Average Number 3.86 3.95 3.91 4.00 3.23 0 0 1.14 1.00 0.14 
Table 8 


Data analyses for 15 Euro-area OECD Series comparing our multiple peak diagnostic with VS, M7, and M8 diagnostics. Our 
multiple peak diagnostic uses the H-FWER method to control the FWER at a =0.05 and a =0.10 and examines the slope at 
= 0.10 (see Section 4.2) 


Data Analyzes - OECD EMU 


Original Data SA Data 
series H-FWER 0.10/0.05 | H-FWER0.10/0.10 VS M7 M8 H-FWER 0.10/0.05 | H-FWER 0.10/0.10 VS 
EMU quartic TH quartic TH quartic TH quartic TH 
PRCNTOO1 345 1345 345 1345 12345 = 0.14 0.44 6 6 6 6 6 
PRINTOO! 12345 12345 12345 12345 12345 0.10 0.23 6 6 p 6 6p 
PRMNCG03 1245 1245 1245 1245 12345 0.12 0.32 l 1 1234 1234 g 
PRMNCSO1 1234 12345 1234 12345 1234 0.22 0.47 g g g 6 6 
PRMNIGO1 12345 12345 12345 12345; 2345 £10:15 0.27 g pb g 6 6 
PRMNTOO1 2345 2345 2345 2345 2345 0.14 0.23 1) v1) l 1) 6 
PRMNVGO1 1234 12345 1234 12345 2345 0.13 0.23 6 6 p 6p 6 
SLMNCNO2 12345 12345 12345 12345 Bi ul ese wis 0.57 6 6 6 ) 6p 
SLMNIGO2 12345 12345 12345 12345 23 0.21 0.45 r) 6 g 6 1) 
SLMNTO02 1345 2345 1345 2345 23 0.20 0.41 24 6 24 34 ) 
SLMNVG02 12345 12345 12345 12345) 152345 28017 0.30 l 1 1245 1345 g 
SLRTTOO1 12345 12345 12345 12345 12345 0.05 0.12 p 6 6 B 6 
SLRTTO02 12345 12345 12345 12345 12345 0.05 0.11 p p 6 6 6 
XTEXVAO1 1345 2345 1345 2345 23 0.31 0.57 6 p 12 12 B 
XTIMVAO1 2345 2345 2345 2345 23 0.40 0.72 6 6 6p 6 6 
Total “correct” 15/15 15/15 15/15 15/15 15/15—«15/15 15/15 12/15 13/15 10/15 11/15 15/15 
Average Number _ 4.40 4.60 4.40 460 3.73 0.20 0.13 0.87 0.80 0 
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Table 9 

Data analyses for 11 French OECD Series comparing our multiple peak diagnostic with VS, M7, and M8 diagnostics. Our multiple 
peak diagnostic uses the H-FWER method to control the FWER at a =0.05 and a =0.10 and examines the slope at 5 = 0.10 (see 
Section 4.2) 


Data Analyzes -OECD FRA 


Original Data SA Data 
series H-FWER 0.10/0.05 = H-FWER 0.10/0.10 VS M7 M8 _~ H-FWER0.10/0.05 | H-FWER 0.10/0.10 VS 
FRA quartic TH quartic TH quartic TH quartic TH 
PRAFAGOI 12345 12345 12345 12345 123 0.13 0.29 g 6 6 g g 
PRNCTOO1 2345 2345 2345 2345 235 0.14 0.44 6 6 g 6 1) 
PRMNCGO01 12345 12345 12345 12345 234 OTS34. 088 l l l l g 
PRMNCSO1 12345 12345 12345 12345 234 0.25 0.58 p 6b b 6b p 
PRMNIGO1 12345 12345 12345 12345 12345 0.11 0.26 g g ) g 6 
PRMNTOO1 12345 12345 12345 12345 12345 0.16 0.29 123 123 123 1234 6 
PRMNVEO1 12345 12345 12345 12345 12845 0,241. 034 g 6 1245 1245 g 
SLRTCRO1 1345 2345 1345 2345 123 027 O71 6b 6 p b 6b 
SLRTTOO2 12345 12345 12345 12345 12345 0.16 0.36 g g g 1) g 
XTEXVAO1 1345 1345 1345 1345 23 0.14 0.44 ) ) 6 1) g 
XTIMVAO1 1245 1245 1245 1245 23 0.18 0.54 6b 6b p p 1) 
Total “correct” 11/11 11/11 11/11 11/11 1041 = T/A I/D 9/11 9/11 8/11 8/11 11/11 
Average Number 4.64 4.64 4.64 4.64 3.55 0.36 0.36 0.73 0.81 0 


Table 10 

Data analyses for 17 Great Britain OECD Series comparing our multiple peak diagnostic with VS, M7, and M8 diagnostics. Our 
multiple peak diagnostic uses the H-FWER method to control the FWER at a =0.05 and a =0.10 and examines the slope at 
5 = 0.10 (see Section 4.2) 


Data Analyzes -OECD GBR 


Original Data SA Data 
series H-FWER 0.10/0.05 = H-FWER 0.10/0.10 vs M7 M8 _ H-FWER0.10/0.05 = H-FWER 0.10/0.10 5 
GBR quartic TH quartic TH quartic TH quartic TH 
PPIAMPO1 1234 1234 1234 1234 24 0.56 1.58 b 6B p 6p p 
PPLAMP02 1 1 123 123 2 0.53 0.92 B 6 p 6p 6 
PPIPFUO1 1345 12345 1345 12345 12 0.64 0.59 B 6 p p p 
PRINTOO1 1345 1345 1345 1345 23 0.16 0.40 6 6p p p p 
PRMNCG02 2345 2345 2345 2345 123 0.23 0.56 6b 6p 6 p 6p 
PRMNCG03 12345 12345 12345 12345 123 0.20 0.49 6p p p p p 
PRMNCS01 123 12 123 1234 i: 0.68 1.31 6b p p 6B 6b 
PRMNIGO1 2345 2345 2345 2345 123 0.15 0.47 I 6b 6b p p 
PRMNTOO01 1345 1345 1345 1345 23 0.17 0.45 6 p p p p 
PRMNVE02 12345 12345 12345 12345 12345 0.25 0.76 6p p p p p 
PRMNVE03 1234 1234 1234 1234 234 0.29 0.91 6p p p p 6b 
PRMNVGO1 124 134 124 134 234 0.18 0.58 p p p p B 
SLRTCRO3 1345 1345 1345 1345 124 0.42 0.74 124 123 124 123 g 
SLRTTOO2 12345 12345 12345 12345 12345 0.05 O15 1234 6p 1234 12345 pg 
UNLVRGO1 12345 12345 12345 12345 1245 063 061 6b p p p 6p 
XTEXVAOI 134 134 1345 1345 23 0.34 1.02 6 6 6 6 6 
XTIMVAOI 23 23 23 23 23 0.31 0.90 6 6 6p 6p p 
Total “correct” 17/17 17/17 17/17 17/17 17 = ATT «14/17—«14/17 16/17 13/17 15/17 17/17 
Average Number 3.76 3.76 3.94 4.06 2.76 0.47 0.18 0.53 0.47 0 
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For the multiple peak-testing scenario, we employ known 
results from the multiple testing literature (ie. the 
applications to controlling FWER) to combine the p-values 
from the five seasonal frequencies in such a way to 
dramatically increase statistical power, as demonstrated in 
Table 4. Although there is some departure in the size (Table 
3) for the multiple peak testing, the results are still quite 
usable. On a typical batch of seasonal series, the number of 
Type I errors are as expected, and the power is quite decent 
(Tables 5-10). While our method compares quite favorably 
to the VS, M7, and M8 diagnostics, neither of the 
diagnostics provide a p-value and only the former can 
distinguish which spectral peaks are contributing to seasonal 
behavior. This aspect is important to the seasonal adjuster, 
who wants to know not only that there may be residual 
seasonality, but also at what seasonal frequencies, so as to 
take appropriate action to alter the seasonal adjustment 
filters (this can be done by smoothing over additional years, 
which is accomplished by changing the seasonal filters in X- 
11-ARIMA; alternatively, one might consider shortening 
the series. For current research on a model-based approach 
to designing SA filters targeted for specific seasonal 
frequencies, see Aston, Findley, McElroy, Wills, and Martin 
(2007)). 

The choice of kernel surely has some impact on the 
results, although we found little difference in practice 
between the quartic and TH kemels; the TH may be 
marginally more powerful. Of course, plenty of other 
popular kernels may also be utilized by a practitioner, and 
we have only chosen two that seemed intuitive and straight- 
forward to implement. The choice of the location p is 
clearly dictated by the characterization of seasonality. Since 
Statistical power generally decreased with B, we always 
recommend taking the maximal £ such that the kemel 
supports are disjoint, which guarantees the asymptotic 
independence property of the various diagnostics that is 
crucial to our multiple testing method. 

Finally, the asymptotic results require that the data be 
differenced to stationarity. Recognizing that economic time 
series are typically nonstationary, it is desirable to trend- 
difference seasonally adjusted data before applying our 
diagnostic. This differencing may dampen the detection of 
the first seasonal peak, so practitioners may “re-color” the 
data as described in Section 3.4. 
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Appendix 


Here we derive asymptotic formulas for the statistical 
measures y , of slope and convexity. These results can then 
be applied in the testing paradigm to get asymptotic critical 
values. Some mild conditions on the data are required for 
the asymptotic theory; we follow the material in Taniguchi 
and Kakizawa (2000, Section 3.1.1). Condition (B), due to 
Brillinger (1981), states that the process is strictly stationary 
and condition (B1) of Taniguchi and Kakizawa (2000, page 
55) holds. Condition (HT), due to Hosoya and Taniguchi 
(1982), states that the process has a linear representation, 
and conditions (H1) through (H6) of Taniguchi and 
Kakizawa (2000, pages 55-56) hold. Assumption 1 (8) of 
Chiu (1988) is a summability condition on various higher 
order cumulants, which is satisfied, for example, by a 
Gaussian process with spectral density in C*. None of these 
conditions are stringent; for example, a causal linear process 
with fourth moments satisfies (HT). The main result is a 
joint convergence of any two measures y ,(/); e.g., these 
can be a slope and convexity measure with the same kernel 
A. We present the general theorem that covers these two 
cases. 


Theorem | Suppose that the fourth order cumulants of {X ,} 
vanish; that either condition (B) or (HT) holds; and that 
Assumption | (8) of Chiu (1988) holds. Let the kernels A 
and B satisfy conditions (i) through (iv) of Section 2.1. 
Then 


oieald) he PA tw 
sepals 
Jn Sa x a ‘ 


=> (Oye) 
of Os CAD 


as n —» ©. Here 0 denotes the zero vector (0, 0)', and V 
isa 2 x 2 matrix with entries 


CAG pat he vi 
00(17)8 pf) 


Proof. First we establish that @ ,,(/*) “20 ,(f”). 
Since the keel A is continuous in an interval (such as 
[u — B/2, up + B/2]), this result follows directly from 
Corollary 1 of Chiu (1988), noting that they deal with the 
Riemann sums approximation to the integral functional 
(Chiu (1988) also defines the periodogram with a 27 
factor). Of course the same results holds with B in place of 
A. Secondly, consider the joint convergence of 8,(/) and 


Vi. = Vy = 1 Vio = Vy, = 
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6,(/). We use the Cramer-Wold device, and apply Lemma 
3.1.1 of Taniguchi and Kakizawa (2000), appropriately 
generalized to include non-even functions (cf. Theorem 3 of 
Chiu (1988)). Hence for any x, y real, 


“at (6,4) = 9,(f)) , , @,W) = om 


foe) Yop) 


a vo = [(Ca)C(-a) + CAS? dh. 
TMe-% 


+ 


using Slutsky’s Theorem (Bickel and Doksum 1977), where 
the kernel C is defined by 


eee 
No) Aah) 


Clearly C(A) C(-A) = 0 and 


Ones 


2 
x 


C0 
ie aru) 


A*(X) 


+2 ioe 


NPG CPC 


Ay Tate xy. 
s 0.) (A) 


A(r) B(A) 


By taking x and y to be zero and one in various 
combinations, we deduce the stated variance matnx V. 

Next we discuss the multiple-peak testing scenario. So 
suppose that we have a finite collection of kernels 4, for 
i = 1, 2, ..., d, each of which satisfies the assumptions of 
Section 2. Then we can easily generalize Theorem 1 from 
two to d kernels as follows. The asymptotic covariance 
matrix V will have ij" entry 


Srna, 
o.(F)0e(f) 


Thus, if the support for any two kernels is disjoint we obtain 
asymptotic independence, and can therefore invoke the H- 
FWER multiple testing procedure. 
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On the definition and interpretation of interviewer variability 
for a complex sampling design 


Siegfried Gabler and Partha Lahiri ' 


Abstract 


Interviewer variability is a major component of variability of survey statistics. Different strategies related to question 
formatting, question phrasing, interviewer training, interviewer workload, interviewer experience and interviewer 
assignment are employed in an effort to reduce interviewer variability. The traditional formula for measuring interviewer 
variability, commonly referred to as the interviewer effect, is given by ieff = deff _int =1+ (Rin —1)Pints Where Pint 
and 7;,, are the intra-interviewer correlation and the simple average of the interviewer workloads, respectively. In this 
article, we provide a model-assisted justification of this well-known formula for equal probability of selection methods 
(epsem) with no spatial clustering in the sample and equal interviewer workload. However, spatial clustering and unequal 
weighting are both very common in large scale surveys. In the context of a complex sampling design, we obtain an 
appropriate formula for the interviewer variability that takes into consideration unequal probability of selection and spatial 
clustering. Our formula provides a more accurate assessment of interviewer effects and thus is helpful in allocating more 
reasonable amount of funds to control the interviewer variability. We also propose a decomposition of the overall effect into 
effects due to weighting, spatial clustering and interviewers. Such a decomposition is helpful in understanding ways to 
reduce total variance by different means. 


Key Words: Interviewer effect; Interviewer workloads; Intra-interviewer correlation; Spatial clustering; Unequal 
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weighting. 


1. Introduction 


A major source of measurement errors in surveys is due 
to the interviewer. This fact was recognized as early as 1929 
by Rice and later by many survey researchers. Factors such 
as the quality of questionnaire design and the interviewer 
can influence the interviewer effects on survey statistics. 

The interviewer can introduce homogeneity in survey 
data, which generally reduces the effective sample size and 
thereby increases the total variance of a survey estimator. 
The within interviewer homogeneity has been traditionally 
measured by the intra-interviewer correlation coefficient 
Pint: Lhe magnitude of the intra-interviewer correlation was 
studied by many researchers, mostly in the context of 
telephone surveys without any spatial clustering effects 
(Kish 1962; Gray 1956; Hanson and Marks 1958; Tucker 
1983; Groves and Magilavy 1986; Heeb and Gmel 2001, 
and others). Researchers have argued that the nature of the 
survey items may affect the value of p,,,. Attitude items 
and complex factual items are considered more sensitive to 
the intra-interviewer correlation than simple factual items 
are (Collins and Butcher 1982; Feather 1973; Fellegi 1964; 
Gray 1956; Hansen, Hurwitz and Bershad 1961). According 
to Groves (1989), values above 0.1 are seldom observed. 
See Schnell and Kreuter (2005) for further discussion on 
this issue. 


As noted by several researchers, the standard interviewer 
effect formula 1+ (7%, — 1)p;,, suggests that even with a 
small intra-interviewer correlation, the interviewer effect 
could be substantial simply due to a high average 
interviewer workload. For example, when p,,, = 0.01 and 
n. = 70 we have ieff = 1.69 (Schnell and Kreuter 
2005). Note that a high average interviewer workload (e.g., 
between 60 and 70) is very common in telephone surveys 
(Tucker 1983; Groves and Magilavy 1986). For the 
European Social Survey, Philippens and Loosveldt (2004) 
provided box plots of the intra-interviewer correlations and 
the interviewer workloads for 18 participating countries. 
The interviewer effect or variance is generally defined as 
the inflation to the total variance caused solely by the 
interviewers. For an epsem design with equal interviewer 
workload, the interviewer variance for the sample mean is 
simply given by 1+(n,, —1)Pix, Where 7, is the 
common interviewer workload. For complex surveys with 
unequal interviewer workload, survey researchers frequently 
use a simple modification of this formula where the 
common interviewer workload is replaced by the average 


interviewer workload, i.e., the formula 1+ (%,, — 1)p;,,- In 
Section 2, we argue that this standard formula 
1+ (7,,, — 1)p,,, Cannot be interpreted as an inflation to the 


total variance caused by the interviewers even for an epsem 
design with unequal interviewer workload. In Sections 2-4, 
we observe that the interviewer variance definition depends 
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on the nature of the complex sampling design and also on 
the interviewer workload assignment. In this paper, we 
provide appropriate definitions of the interviewer variance 
in different survey scenarios. A reliable definition of the 
interviewer variance is helpful in determining actions that 
need to be taken in order to reduce interviewer variability. 
This paper is foremost applicable to the planning of surveys 
rather than analyzing survey data. In other words, in this 
paper we have concentrated on the definitions and 
interpretation of the interviewer variability and not on 
estimating it from a given survey. 

In Section 2, we consider an epsem design with no 
spatial clustering and provide a model-assisted interpretation 
of ieff. We show that for the equal interviewer workload 
ieff is simply the ratio of the variances of the sample mean 
under a correlated model that accounts for the homogeneity 
of the observations collected by the same interviewer and a 
simple uncorrelated model that fails to account for such 
homogeneity. Thus, multiplying the variance of the sample 
mean for simple random sampling by the ieff one can 
obtain the total variance of the sample mean _ that 
incorporates both the sampling and the interviewer 
variability. This is a very intuitive interpretation of ieff and 
complements the model-assisted justification given earlier 
by Kish (1962). In this section, we also show that for an 
epsem design ieff is lower than the model-assisted 
interviewer effect formula if the interviewer workload varies 
and the intra-interviewer correlation is positive. Thus, the 
survey designer who uses ieff would give less effort to 
control interviewer variability than is really needed. In this 
situation, an appropriate interviewer effect formula can be 
obtained from ieff when a weighted average interviewer 
workload is used in place of the usual simple average. 

In Section 3, we entertain the possibility of unequal 
weighting but no spatial clustering. We obtain a model- 
assisted interpretation for ieff if and only if the respondents 
interviewed by the same interviewer share the same 
sampling weight and the interviewer workload is inversely 
proportional to the square of the common weight for the 
interviewer. Interestingly, unlike the epsem design, equal 
interviewer workload does not necessarily guarantee a 
model-assisted interpretation for ieff. When there is an 
equal interviewer workload and there is at least one 
interviewer for which the respondents do not all share the 
same sampling weight, we show that ieff is always higher 
than the model-assisted formula. We also point out the 
factors that cause the difference between these two 
formulae. These results have a practical relevance in terms 
of saving survey costs. To be specific, the survey designer 
who uses ieff is likely to allocate more funds to control 
interviewer variability than is really needed. We have also 
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cited some situations where ieff could have an under- 
estimation problem and thus survey designers who use ieff 
could give less emphasis to control the interviewer effects. 
Our formula provides a more accurate assessment of 
interviewer variability and thus is helpful in the allocation of 
more reasonable amount of funds to control the interviewer 
variability. Furthermore, the change in planning formulae 
will affect the sample size. 

In many large scale sample surveys, due to various 
organizational and financial reasons such as the absence of a 
general population register or to reduce the overall survey 
costs, a multi-stage clustered sampling design is considered 
to be a cost-efficient alternative to simple random sampling. 
Under a multi-stage clustered sampling design, respondents 
who live in close spatial proximity of each other get 
selected. Respondents living in the same spatial cluster tend 
to share similar attitudes because of their similar socio- 
economic background and hence increase the internal 
homogeneity of the survey data. This spatial homogeneity 
violates the iid (independently identically distributed) 
assumption frequently used in standard statistical inferential 
procedures and so does the clustering within the 
interviewers. This fact has been recognized by many survey 
researchers and adjustments to various statistical procedures 
and the related software issues have been addressed in the 
literature (see Rao and Scott 1984; Skinner, Holt and Smith 
1989; Biemer and Trewin 1997; Chambers and Skinner 
2003; among others). In Section 4, we present a new 
definition of the interviewer variability in the presence of 
unequal weighting and spatial clustering. In the presence of 
spatial clustering, we argue that ieff generally has a 
tendency to overestimate the interviewer variability. Thus 
for complex surveys involving spatial clustering, ieff may 
unnecessarily give a false alarm regarding the magnitude of 
the interviewer variability. 

In Section 5, we discuss the effects due to the combined 
effects of weighting, spatial clustering and the interviewer. 
The formula for overall effects offers an accurate 
determination of the sample size at the planning stage. We 
provide a nice factorization of the overall effects into the 
effects due to weighting, clustering and interviewer. Such a 
decomposition of the overall effects can be useful in 
understanding ways to reduce the total variance by different 
means. In discussing Verma, Scott and O’Muircheartaigh 
(1980), Hedges mentioned the need for such an overall 
effect formula. We generalize a formula earlier proposed by 
Davis and Scott (1995) to a non-epsem design and for a 
general correlation model valid for both discrete and 
continuous data. We present proofs of all the technical 
results in the Appendix. 
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2. EPSEM design with no spatial clustering 


Let y, denote the observation obtained from the k™ 
respondent interviewed by the i" interviewer 
etek ory eine, ne tne total 
sample size, ¥ = 1/n /_,>j, y,, the unweighted sample 
mean, and 7,,,(a) = >/_,a,n,, a weighted average of the 
interviewer workload, where a, is an arbitrary weight 
attached to the i” interviewer workload and a = 
Cs et 3) 

We shall first provide a model-assisted justification of the 
traditional interviewer effect formula, ie, ieff =1+ 
(7,.—1)Pin> Where 7,,, is the unweighted average of 
interviewer workload. Note that 7, =7,,(a)), with 
Apt dn tet, Ogy sy Gy, — Lid andi iey = lefp(a,): Using 


Result 1 given in the Appendix, we get 


Vaty, (VY) _ 


ALAC Hg meas eer 


| a [Fins (a, ) in Mpa 


where a, = (@,,, .... a), with a, =n,/n. In the above, 
Vary, (y) and Var, (y) are the variances of y under the 
following two models, respectively, 


o ifis=i,k=k', 
M;: Cov(vin, Vix) = 
0 otherwise, 


o ifi=i,k=k', 
MACOVO p30) an Pio ll P= ty heK, 


0 otherwise. 


Note that unlike model M,, model M, introduces 
homogeneity of the observations collected by the same 
interviewer. 


Remark 2.1: It follows from the corollary to Result 1, given 
in the Appendix, that for p,,, > 0, ieff(a,) = ieff if and 
only if n, =n/I for all i, ie, if and only if each 
interviewer has the same workload. For the balanced case, 
Kish (1962) provided a model-assisted justification of ieff 
using a linear mixed model, which is a special case of M,. 
For the unbalanced case, it is interesting to note the 
similarity between the interviewer variability formula 
ieff(a,) and the design effects formula given in (A3) of 
Holt in discussing Verma ef al. (1980). 


Remark 2.2: It follows from the corollary to Result | that if 
p.,, > 9 and n,’s are not equal then ieff(a,) > ieff. 


In the following example, we demonstrate the extent to 
which ieff(a,) and ieff could differ for different 
interviewer workload patterns. 


87 


Example |: In Table 1, we consider three different workload 
assignments for ten interviewers, each with n = 790. Case 
A) represents the most variable workload assignment with a 
standard deviation = 68.3; Case B) is nearly balanced with 
a standard deviation = 9.5; Case C) corresponds to the 
equal interviewer assignment. 


Table 1 
Three different interviewer workload assignments (Example 1) 


Interviewer workload pattern 


Interviewer A) B) C) 
] 4 70 79 

2 10 70 79 

3 20 70 79 

4 34 70 79 

5 52. 70 79 

6 74 88 79 

7 100 88 79 

8 130 88 79 

9 164 88 79 

10 202 88 79 

n 790 790 790 
Tin (21) 132 80 79 


Let ieff(a,.,), ieff(a).,), and ieff(a,..) = ieff denote 
ieff(a,), the model-assisted interviewer variance formula 
corresponding to the cases A, B and C, respectively. For 
Pin > 9 the function ieff(a,) 1s Schur-convex, which 
explains the fact ieff(a,.,) 2 ieff (a).,) 2 ieff(a).-) = ieff. 
Figure 1 provides the values of the interviewer variance 
obtained from the standard formula (i.e., ieff) and our 
model-assisted interview variance formula for all 
combinations of the two influencing factors, i.e., weighted 
average of interviewer workload and the intra-interviewer 
correlation. From Figure 1, it is interesting to note that ieff 
could underreport by about 100%. 


ieff (a) Xe 


Interviewer A: Dashes, 
Interviewer B: Spaced dots and 
Interviewer C: Solid line 


Figure 1 A graph of ieff(a,) vs. pj,, for different 7;,,(a,) 
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3. Unequal weighting with no spatial clustering 


In this section, we consider the situation when we have 
unequal weights. Let w, be the survey weight attached to 
the k™ respondent interviewed by the i" interviewer. In 
this situation, a weighted mean Y=) >,.W,V_/ Di De Wx 
is a popular estimator of the finite population mean (See 
Brewer 1963; Hajek 1971) and the model-assisted 
interviewer variance formula is given by 


5(Em) 


Vary, (,) i iF 
Re eer yee ahh. eee ger 

Vary, (7,,) » is Wi 

Tink 

See Result 1 given in the Appendix. 

Define w, = 1/n, X7,w,, the average survey weight 
for the i” interviewer and o; =1/n, Y,w,-w, the 
variance of the survey weights for the i" interviewer. It can 
be shown that 


Ww 


iefmaila ps (Ae 3) 


where 


Note that, in general, jeff, cannot be written in the form 
eff, = +p. (7, (a) 1) with: Dal. 


Remark 3.1: From Result 2 in the Appendix, we have 
ieff,, < ieff(a,), 
where 


a, = (@,,, .-.. 4,), with a,, = 


In the above, for p;,, > 0, ieff,, = ieff(a,) ifand only if all 
Go; are zero. Thus, ieff (a,) can be interpreted as a 
conservative interviewer variance. 

Equality holds if and only if w, = W, forall i and & in 
which case 


ieff,, = ieff (a3), 


where 


* * * . * 
a, = (a), ..., a), with a,, = 


Thus, the formulae ieff,, and ieff(a,) are equivalent if and 
only if the survey weights are all the same for a given 
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interviewer. One example of such a design is an epsem 
design for which we have 


i 


* 
Ge 
and 


ieff nix ieff (a5) = ieff (a, ). 


Now we shall try to understand the factors that explain 
the difference between ieff,, and ieff. To this end, define 


_— I j lt I = 
Wale ee eee 


weight for all interviewers, 


the average survey 


SSB=>/_,n,(w,—)’, the between interviewer sum of 
squares of the survey weights, 


SSW =>), Ti. (w, -W) =S/.n,0o7, the within  inter- 
viewer sum of squares of the survey weights, 


SST =SSB+SSW, the total sum of squares of the survey 
weights, 


t,, = SSW/SST, an indicator of the relative contribution of 
the within interviewer variability of survey weights to the 
total variability, 


CV,,= JSST/n/w, the coefficient of variation of the 
survey weights in the entire sample. 


It can be shown that (see Result 4) 


ieff,—ieff 


Hine 
SST +nw | 


~ (4CV2) SST 


{i 
n. pons 
>( -1} om 


> edits OV 


a (7, = 
actos Ez -1) na “S50 (2) 
i= int 


1 Pin: (3) 


Remark 3.2: We can use formula (1) in any situation. For 
epsem designs, we have 

I 

>(2 e ie 

i=1\ Mint 


Note that an application of the Cauchy-Schwarz 
inequality suggests ieff,, — ieff > 0 with equality if and 
only if n, = n/TJ for all i. 


ni. 


ieff,, oa ieff = Pint 


int 
n 


Remark 3.3: We can use (2) if SST # 0, i.e., if the design is 
not epsem. If p,,, > 0, (2) implies 
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i=] 


I 
ieff,,—ieff <0 if and only if a < SSW. 
Rint 

If high interviewer workload tends to be associated with 
small average survey weights and vice versa and SSW # 0, 
we can expect ieff to be a conservative value of the actual 
interviewer variance ieff,,. In Example 2, c) and d), we 
have such a situation. 

Now, we have ieff,, = ieff if and only if w, = Ww. (or, 
equivalently, SSW = 0) and n,w?/>,n,w? =1/1 for all i 
and k, ie, ieff, =ieff if and only if w, = WwW and 
wo 1/.fn, for all i and k. 

Thus, for a non-epsem design, equal interviewer 
workload does not necessarily provide us a model-assisted 
interpretation for ieff. For example, if the survey weights 
vary within at least one interviewer, we will not have a 
model-assisted interpretation of ieff. Obviously, for an 
epsem design the two formulae are equivalent if and only if 
we have equal interviewer workload. 


Remark 3.4: If the interviewer workload is the same for all 
interviewers, we have 


Table 2 


89 


Tie 


int “w 


Bh, — 16 = Ta 


yi Pint 


(assume SST # 0). Thus, ieff is a conservative value of the 
actual interviewer effect ieff,. Furthermore, |ieff, —ieff| is 
an increasing function of the common interviewer workload 
7m, and t,/(1+CV,”) (for fixed CV~, the latter is an 
increasing function of t,,). The same interviewer workload 
is given in Example 2 a). 


Remark 3.5: We can use formula (3) if SSW > 0, ie, if 
there is at least one interviewer for which weights are not all 
equal. 


Example 2. 


Table 2 presents eight different combinations of 
(n,, W,, 6; ). The first combination assumes equal n, values 
but unequal weights. The second combination assumes 
Ww, o 67. The other six combinations show all possible 
ordering of 7,,,, %,,(a,), %,, (a) and, therefore, ieff, 
ieff(a,), ieff,, ieff(a,) taking into consideration that 
ieff < ieff(a,) and ieff, < ieff(a,). 


Ordering of interviewer effects formulae for several parameter combinations (Example 2); in the 


last column p,,, = 9.01 


ieff = ieff(a,) = ieff(a,) > ieff, 


uy W, o; Ting Mine (81) n, 
a) eel L022) 10299 2S 25 19.20 
25 1.0365 9 0:375 
Zo 01998 0.276 
25 0.945 0.260 
b) 10 1 1 25 30 15 
20 1 1 
30 1 1 
40 1 1 
Cc) LO 1 1 25 30 1) 
20 1 2 
30 1 3 
40 1 4 
d) 10 1 4 25 30 10 
20 1 3 
30 1 2 
40 1 1 
Qy lv 4 144 ZS 30 1.80 
20 2 9 
3 OR O333 0.555 
40 0.250 0.125 
De OP e03335 0.025 MS 30 31.82 
20 0.666 0.075 
30 1 0.125 
40 = 1.333 0.175 
2) lO 1 0.010 25 30 ZO 
20 1 0.020 
30 1 0.030 
40 ] 0.040 
h)aaeO 1 0.004 as 30 29.94 
20 l 0.003 
30 1 0.002 
40 1 0.001 


Thy, (A) Interviewer effects ieff /ieff, 
25 1.003 
30 iff, < ieff < ieff(a,) = ieff(a,) 1.007 

32.5 eff, < ieff < ieff(a,) < ieff(a,) 1.023 
26.7 _ eff, < ieff < ieff(a,) < ieff(a,) 1.015 
11.71 eff. < ieff(a,) < ieff < ieff(a,) 0.998 
35.26 —iieff < ieff(a,) < ieff, < ieff(a,) 1.015 
30.10 eff < ieff. < ieff(a,) < ieff(a,) 0.999 
29.99 eff < eff. < ieff(a,) < ieff(a,) 0.998 
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In the example, );n,W, = n. We now explain the eight 
different patterns. 


a) Since all n, are equal, ieff = ieff(a,) = ieff(a,). 
Moreover, ieff, is smaller than the rest because of 
the fact that o7 > 0. 


b) Since o; are relatively large, ieff,,<ieff. Also, 
Oo; = c- w implies ieff(a,) = ieff(a,). 


c) Since o? are relatively large, ieff,, < ieff. Moreover, 
since W +0; and n, are both increasing, we have 


ieff (a,) < ieff(a,). 


d) Since o/ are relatively large, ieff, < ieff. Since 
W- +7 is decreasing and n, is increasing, we have 


ieff (a;) < ieff(a,). 


e) Since o, are relatively large, ieff,, < ieff. Also, 
w, and o, are decreasing and n, is increasing 


implying ieff(a,) < ieff(a,). 


f) The fact that # and n, are increasing implies that 
ieff,, > ieff; since o; and n, are both increasing, 
we have ieff(a,) < ieff(a,). 


g) Since ww and n, are increasing, we have 


ieff,, > ieff and since oO, is increasing, we have 
ieff (a,) < ieff(a,). Moreover,  ieff, < ieff(a,) 


since o, is smaller than that in f). 


<=) : : a 
h) Since w; and yj are increasing, we have ieff, > 
ieff and since o; 1s decreasing, we have ieff(a,) < 


ieff (a, ). 


4. Unequal weighting and spatial clustering 


In this section, we obtain an appropriate interviewer 
variance formula in the presence of spatial clustering and 
unequal probability of selection. Consider the situation 
when more than one interviewer work independently in the 
same psu and the respondents in each psu are randomly 
assigned to the interviewers. We shall assume that no 
interviewer works in more than one psu. Such a design was 
considered in Biemer and Stokes (1985). Now we shall 
separate the interviewer effect from psu effect (i.e., spatial 
clustering) and unequal weighting. Let y,,, and w,, be the 
observation and the associated survey weight for the k" 
respondent in the p'™ psu interviewed by the i” 
mtemiewen. (p= |, a Poe PINS | rw Fs n,;). Let 
n, = pa n,, be the number of sampling units in psu p. 
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In this case, we use the following weighted average to 
estimate the finite population mean: 


Pe pes 
Dany on 


— _ p=lizl k=l 
Vay P Lp Poi ; 
> 2d, DoW pte 
p=) i=1 k=l 
Define 
< Vary, (Y,,) 
le Ww | are ee 
Vary, O,) 


where the suffixes s and w signify the presence of spatial 
clustering and unequal weighting. In the above, Var,, (¥,,) 
and Var,, (¥,,) are the variances of y,, under the following 
two models respectively 


o if p= Perel, ken 
M;: Cov(Yp> Voie) =\PcO if p=p’, k#k’ 
0 otherwise 


5 if p=p, i=i, k=k 

if p=p, TAT 

M:;; Cov(y nie» Yorn’) me : ” ’ 
po = if p=p’, isi’, k#k 
0 if p¥p' 


In the above, p- is the intra-psu correlation and p is the 
combined interviewer and psu intra-class correlation. Define 
Pint = P—Pc, intra-interviewer correlation. Usually, p,,, > 0. 

From Result 5, we have 
Fin (Ay) — 1 


le sw It Cine. be eae 
l € Po M%pgu (D,,) A 1) 


where 
—2 
Ni Wi 
A, = (Cee) is and Api ri Je Mrs Tis 
pal ..-5 z 
»S W pik 
p=l i=l k=l 


with 
IVE 2 
eal =e Wks 
N yi k 
Poli e tl 1G : 
Ww. 
per ye >, pik 


23 p=l i k 
TAN De On es ty or ay 
Avert: 2 
» Woik 


and 
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nyW, 
b,, = (bg) ear th and by» ms P i Np, 5) 
2 
yy W vik 
p=l i=l k=l 
with 
1] His ney i ie 
gar: > pik ‘a Side pi? 
Ny i=l k p i=l 
and 
aif be 1g : 
P Depoi damn 
ip p=l ak 
pan (D,,) Dy wp! p as 
1 


Note that 7,,,(A,,) S 7,,,(b,,) with equality if and only 
if J, =1. Also note that 7,,(A,,) is invariant of the 
alldcatt oh of the interviewers to the psu’s while 7... (b,,) 1s 
not. 


psu 


Remark 4.1: If p. =0 we get 
ie S,w = | = Pint (7, (A,,) aa 1). 


This formula is similar to ieff,, given in Section 2. Thus, 
AM the comments given in Remark 2.1 apply here. Note that 
7, (A,,), Just like My cannot be generally written in the 
form 7,,,(A,) = £1 4yyiN,, With a Sie Ayn = 15 
the same comment applies to 7... (b,,). 


Remark 4.2: Define 
Phul, 
ne (A)= >), 4p where A=((a,,)), with a,, = 
p=! i 


and 


P 
Frou b)= Db My where b = (8,,..., bp) with b, =— 
p= 


If p. #0 but we have an epsem design, then we drop the 
suffix w in ieff, ,. Note that 


A)-1 
faggot A ip prssen ath) Tibet 
1+ PcLMs.(b) — 1] 


Jos. Pint ‘ Fins (A) oA : Pc (A,,, (db) ray 1) 
Pc Nosu (b) salts dit Pc Wie (b) = 1) 
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so that 


ieff. rey Pf rg eg ei Pine (A) 
Pc "osu (b) —1 Pc Nosu (b) 


It can be readily seen that the right side of the inequality 
increases with the ratios p,,,/Pp¢ and 


7 (A) -1 


Hint 


Fi, (b) — 1 
We have 


Pins (A Al —[l+p- Wioai (b)—1)] 


ieff, —ieff =Pin ee Gehan =) 


Thus, for p,,, > 9, 


ieff,<ieff if and only if 
Deff, = 14D Fyy(b)=1) > 
int 
e., if and only if the design effect due to the spatial 
clustering is larger than the ratio of the weighted average of 
the interviewer workload —1 and the average interviewer 
workload —1. If the interviewer workload is the same for all 
the interviewers, the nght hand side of the inequality is 1 
and so the inequality is always valid. It is interesting to note 
that jeff ~4-ief. if p,, =O0.lep. =0.05, (b)=140, 
and 7, = 70. 


Nysu 


<3 (Tine Li »| 


Remark 4.3: In the general case, we have 


Fin (Ay) = 1 
is ieff Ti Pint ae 
Lia Po (Mosu (b,,) az 1) 
Thus, for p,,,, > 9, 


ie < ieff if and only if 


S,w 


n.(A.)-1 
Def, = 1+ PeF(b,,) — 1) > es 
Neat on 
i.e., 1f and only if 
a) GA yea 4 
Pc > Min (Ay) = Mine =! Pc, Say. 


Vine 01, 0,,) =D 


In Example 2 (see Table 3), ieff is a conservative value for 
, for a) to e) if p. > 0. The same holds for f) to h) if 
Pc > 0.004. 
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Table 3 
Average interviewer workloads for several parameter combinations (Example 2); ieff /ieff,,, for p,,,= 0.01 and p. = 0.02 
IA = (1,3) IA = (2,2) IA = (3,1) 
= hs i was : ieff pe a Is ieff * s ‘ ieff 
n; WwW, ro n int n int (A, ) Aas (b,,) Pc ieff. . n int (A, ) Assu (b,,) Pc ieff. e n int (A,) Ayu (b,,) Pc ieff. x 
a) 25 1.022 0299 25 19.202 47.528 0.005 1.133 19.202 38389 -0.006 1.123 19.202 49.039 0.005 1.135 
25 1.036 0.375 
25 0.998 0.276 
25 0.945 0.260 
biG! l 25 15 Al 0.010 1.151 15 29 0.015 1.138 15 26 0.017 1.134 
20 Sigel I 
30 I 1 
40M y l 
c) 10 ie" 1 I 25 mS 20,500 057 UP T185 75 145 0.054 1.180 75 13 0.061 1.178 
261 2 
30 <j 4il 3 
40ey, I 4 
d. 16% 1 Ai) OS 10 27333 H.024 TAT! 10 19.333 0.034 1.163 10 17.333 0.038 1.161 
20: just 3 
201 al 2 
40 get 1 
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30 fot > 0H25 
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g) 10 1 0010 25 29.126 79.612 0.002 1.118 29.126 56311 0.003 1.094 29.126 50485 0.003 1.086 
20 ney Her 's0.020 
30019" 70,030 
40 1 0.040 
h) 10 1 0.004 25 29940 81.836 0.003 1.117 29.940 57.884 0.004 1.092 29.940 51.896 0.004 1.084 
207 Le? 0003 
30 strat 7f 0.002 
40 1 0.001 
If a household and a person within the household are Furthermore, 
selected at random, then the weights are often independent bere 2 a Snist 2 
. . P pi P 
of the psu and the interviewer and depend only on the > W vik Sy [Sm ‘ 
3 2 c Ue Py. 
household sizes. In such a situation, the household sizes Nae p=l i=l \ k=l _ p=lizl\ j=l 
form the weighting classes. For weighting classes, we define ‘ahha ee gl ys hme 
Ss Woik wm, 
paliel ha i! 
m,,: number of sampling units in psu p assigned to and 
interviewer i belonging to weighting class /, yee 2 ‘ 
I, : ‘arias 3 2 
m,, = %;2,m,,: number of sampling units in psu p | OL W nik 2(L¥imy 
belonging to weighting class /, 77,,,(D,.) = a ae - é 
I a 2 : ae 
m, = Xp-1di2:m,,: number of sampling units belonging to ys Wik wim, 
weighting class /. pal it Ee j=l 
are ratios of quadratic forms in w = (w, ..., W,). 
Thus, 
n,; = Lj-m,,: number of sampling units in psu p 5. Overall effects 


assigned to interviewer i, 
I : oi ie 
ny, = din fa) yy: number of sampling units in psu p, 


PP se oJ ; : 
n= Dp-Lin1Lj-1M,,: sample size. 
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The overall effects take into account unequal weighting, 
spatial clustering, and the interview effects and can be 
viewed as a generalization to the traditional design effects. 
Multiplying the SRS variance for the unweighted sample 
mean by the overall effects will provide the total variance 
estimator. 
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V = 
EO a 
where 
ff Var,/- (¥,,) 
Spe? 
Vary) (y ) 
V 2 
= ary (Fw) 
Var, /- (¥,,) 
Vare ly, 
Mme = 1 Bow = vas ; 
M, w 


In the above, Var,.is with respect to the following 
model: 


ME ree P 

. fi = las k=k 

MAG OO ey Sane F Pisa! . 
Opus Yove) ‘ otherwise. 


It can be shown that 


P fy Mi 
Day ADaptn 
=Wi=lik=1 
Gf=— 5 
P ihe Ni 
> 
p= i=l k=l 
P (ong 2 Palen Bei : 
Wik Da, Dit 
p=l\ i=l k=l joel k 
x] 1+ sal iylc 15> = 
Pc P Us i h Pint P It Ny 5 
Dy Wik Ds W pik 
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The relative contributions of weighting, spatial clustering, 
and interviewer effects to the overall effects are given by 


Ny; 


iP ibe 
2 
md 22a i 


p=l i=l k= 
PL, Ny 4 
ESF 0 
Re 
Ail off 
1+ n...(b.)—-1 
Reeff. = pet baal >) ) 
eff 
a ee a 
Deer ei (bay cal) 
Reef, = chaste dalle 


In Figure 2, we present three dimensional graphs of the 
relative contributions of weighting, spatial clustering, and 
interviewer effects to the overall effects for different 
combinations of intra-cluster and intra-interviewer correla- 
tions for different patterns of weights given in cases a), f) and 
h) of Table 3 with JA = (1, 3), where JA = (a, b) indicates 
that the first a of the four interviewers are in psu | and the last 
b interviewers are in psu 2. 
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Remark 5.1: From Result 6, we get 
n n 
eff =1+ —-l|+p,,);——-1}. 
sis Pc (= Pint - 


The night side is the overall effect if the same number of 
interviewers with equal workload is assigned to each psu. It 
is interesting to note the similarity between the right hand 
side of the above inequality and the design effects formula 
given in (3.1) of Hansen, Hurwitz and Madow (1953, Vol. I, 
page 370). To claim the similarity, we need to treat the 
secondary sampling units as the units belonging to an 
interviewer. In this connection, we also note the formula 
(3.7) given in Hansen et al. (1953, Vol. II, page 292) for the 
casetl = ar 


Remark 5.2: When we have the same weighting classes 
across psu x interviewer, we have 


Remark 5.3: Consider the special case 
n 

in which we allow variation in weights within psu x 
interviewer classes, but we constrain the weights to have the 
same relative frequency distribution in each class, i.e., the 
means and the variances of the weights within the classes do 
not depend on the class (Lynn and Gabler 2004). It is easy 
to see that in this case 
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Figure 2 Relative contributions of weighting, design and interviewer effects to the overall effects for cases a), f) and h) in 


Example 2 for the case JA = (1, 3) 


Using the same argument given in the proof of Result 6, 


we get 


Offa le pr, % 
p=) i=1 


ee Pc (Fis (b) ely Pint (Hin, (A) 1). 


This means that the overall effect is larger than the 
overall effect for an epsem design (see Remark 5.4). 


Remark 5.4: For an epsem design, we have 
eff = l + Pc (Pic (b) ry; 1) a Pint (Tin (A) ry 1), 


where 
P je Ne 
Li" pa pay 
7i,,,(b) = =— and 7,,(A) = =+—. 
n n 
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Note that Davis and Scott (1995) obtained this formula 


for the special case of the following linear mixed model: 


Y pik a H+ Q,; tp + Ely, 


where is the overall effect, a,,B, are random effects 


due to the interviewer i, psu p and €,, is the pure error. 
They assumed that the random effects are independent with 
a, ~ N(0,0%),B, ~ N(0,o5) and a N(0,02). 
For the above linear mixed model, it is easy to check that 
oc. 
B 


and p, = 


Q 
en 


2 2 2 2 
+c Oo, +0, +0, 


no- ~) 


Pint 2 
Oe et Cy 


However, it is instructive to note that the definition eff does 
not require p,,,and p.to be strictly positive and the 
definition goes beyond the linear mixed model. For 
example, the definition applies to the following example: 
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Example 3: A simple model for binary data. 


Assuming 0<min(a, B)<®@< 1, we define the following 
model: 
For all n,, different respondents of interviewer i in 


psu p. 


PY pix = X41 Voigt = X2) 


For all 1,, respondents of interviewer i and psu p and 
all 1, respondents of interviewer i’ and psu p. 


PU ik = 4.1 pie =) 


For all n, respondents of psu p and all 7,, respondents 
of psu p’. 


POs =X, Yovitk! =X) 


Therefore, we have 
E(Y,,) = 8 forall p, i, , 


Var (Y 


P 


4) = 80-6) forall p, i,k, 


Cov (Von Yt’) 


P [Wat You) Var Yyy:) 


2 


_ Soe OT for all p, i and k # k’, 
6(1 — 6) 


Cov (Yoins Yin) 
Var (Y,x) Var (Yin) 
B - 6° 


= ——— forallp and i#/i/’, 
6(1 — 6) 


Pc 
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which is a special case of Model M, with o* = 
Var(Y,) = 9(1 — 8). Note that both p. and p may be 
negative and p;,, =P-—Pc¢ is positive if and only if 
ai>'B. 


Remark 5.5: For an epsem design with common psu size 
b = n/P, wehave 


of =V+ pelo —1) +p. 1.4) — D: 


Remark 5.6: In discussing Verma etal. (1980), Holt 
considered the case when there is no interviewer variability 
and psu is the weighting class, i.e., the case when p,,, = 0 
and w,, = w, forall p, i, k. In this case eff reduces to 


Note that the above formula can be obtained from 
equation (A4) of Holt in discussing Verma ef al. (1980), 
after correcting an obvious typo (i.e., deleting m in the 
denominator), choosing his choice of survey weight and 
some algebra. Design effect formulae in the absence of the 
interviewer effects were considered by many authors. See 
Kish (1965), Verma et a/. (1980), Skinner (1986), Valliant 
(1987), Skinner etal. (1989), Gabler, Hader and Lahiri 
(1999), Lynn and Gabler (2004), Kalton, Brick and Lé 
(2005) and others. 


6. Concluding remarks 


We have noticed that the standard interviewer effects 
formula could have either an overestimation or under- 
estimation problem depending on the situation. For 
example, it could severely underestimate the interviewer 
effects in an epsem sampling design with different 
interviewer workloads. Interestingly, spatial correlation can 
turn this underestimation to an overestimation. In the former 
case, the survey designer who uses the standard interviewer 
effect formula may pay little attention to control the 
interviewer effect. In the latter case, a high value of the 
interviewer effect may unnecessarily raise concerns about 
the quality of data connected with the interviewer. This may 
trigger allocation of a higher portion of budget than is 
necessary to reduce the interviewer effect, which may be 
already much lower than the value obtained by an 
application of the standard formula. The paper is an attempt 
to define and interpret interviewer effects that are 
appropriate in different complex survey situations. 
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We have considered the case when an interviewer is 
assigned only in one psu. The case when an interviewer 
works in different psu’s is also important and will be 
considered in a later paper. The weights used in the 
proposed formulae only account for sampling weights as 
they are planned at the design stage, but do not necessarily 
reflect the actual weights attached to each case once the data 
are collected. In other words, our interviewer effect 
formulae do not incorporate the effects due to nonresponse 
and post-stratification adjustments. The formulae presented 
in the paper are mainly useful in the planning and design 
stage when we have some ideas about the intra-interviewer 
and spatial correlations. 

Reliable estimation of p,,, and p, 1s important. Although 
there are some papers that deal with the estimation of 
P;,, and p,, there is certainly a need to advance research in 
this important area. In comparing the two sources of 
homogeneity, Hansen etal. (1961) found that the 
interviewer variability was often larger than the sampling 
variability. In many surveys, such an evaluation, which 
requires estimation of the intra-interviewer and intra-cluster 
correlations, is either difficult or even impossible because 
the interviewer effects are often confounded with the spatial 
clustering effects. The use of an interpenetrating design, first 
proposed by Mahalanobis (1946), where respondents are 
randomly assigned to the interviewers, is a way to get 
around the problem. In practice, the implementation of such 
a design in a large scale sample survey is difficult, but some 
approximated interpenetrated designs can be applied 
(Hansen etal. 1961, Bailar, Bailey and Stevens 1977, 
Bailey, Moore and Bailar 1978, Collins and Butcher 1982, 
O’Muircheartaigh and Campanelli 1998). Multi-level 
models have been used as a partial remedy to the problem 
(Hox and De Leeuw 1994, Davis and Scott 1995, 
O’Muircheartaigh and Campanelli 1998, Scott and Davis 
2001). We have not considered the problem of the esti- 
mation of the intra-interviewer and intra-cluster correlations. 
This is an important problem and will be considered in a 
later paper. 

In practice, interviewer or design effects are computed 
for many items using the same formula and a summary 
measure such as the median interviewer or design effect is 
taken for the planning and design of the survey. So far as the 
issues related to handling multiple items are concerned, one 
may continue to follow one’s own protocol; the only change 
we may suggest is to use our new definitions for interviewer 
effects or overall effects whenever applicable. The use of 
our formula may suggest overall effects, which may be 
much lower than the standard formula. This, in tum, may 
suggest lower sample size and hence may save survey costs. 
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Corollary: Assume p,,, > 0 and w, = 1/n. Using Result | 
and the Cauchy-Schwarz inequality, we get 
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Result 2. ieff,, < ieff(a,), where 


a, = (a,,,.... &,) witha,;, = 


Proof: Using the Cauchy-Schwarz inequality, we have 
(2a ) Ss aa 
i k i k 


with equality if and only if w, = Ww, for alli and k, where 


is the average survey weight for the i" interviewer. Thus, 
we have ieff,, <1 + [Fy (2) — 1 Pin = teff (ay). 

The equality holds if and only if w, = w, for alli and k 
in which case ieff,, = ieff(a,), where 


* * * . * 
a, = (a, .... @,), with a,, = 
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If all weights are non-negative, then 
1 
GF, = yy aw Sw, 
Nn; k 


since o; is Schur-convex. Defining 
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with equality if and only if o7 = (n, —1)W for all i or if 
all n, are equal. 

The inequality follows from the logarithmic concavity of 
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Now the result follows using algebra. 
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with equality if and only if the weights are all equal and 
each interviewer has the same workload. 
If we have in each psu only one interviewer, then 
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Indicators for the representativeness of survey response 


Barry Schouten, Fannie Cobben and Jelke Bethlehem ' 


Abstract 


Many survey organisations focus on the response rate as being the quality indicator for the impact of non-response bias. As 
a consequence, they implement a variety of measures to reduce non-response or to maintain response at some acceptable 
level. However, response rates alone are not good indicators of non-response bias. In general, higher response rates do not 
imply smaller non-response bias. The literature gives many examples of this (e.g., Groves and Peytcheva 2006, Keeter, 


Miller, Kohut, Groves and Presser 2000, Schouten 2004). 


We introduce a number of concepts and an indicator to assess the similarity between the response and the sample of a 
survey. Such quality indicators, which we call R-indicators, may serve as counterparts to survey response rates and are 
primarily directed at evaluating the non-response bias. These indicators may facilitate analysis of survey response over time, 
between various fieldwork strategies or data collection modes. We apply the R-indicators to two practical examples. 


Key Words: Quality; Non-response; Non-response reduction; Non-response adjustment. 


1. Introduction 


It is a well-developed finding in the survey methodol- 
ogical literature that response rates by themselves are poor 
indicators of non-response bias, see e.g., Curtin, Presser and 
Singer (2000), Groves, Presser and Dipko (2004), Groves 
(2006), Groves and Peytcheva (2006), Keeter et al. (2000), 
Merkle and Edelman (2002), Heerwegh, Abts and Loosveldt 
(2007) and Schouten (2004). However, the field has yet to 
propose alternative indicators of non-response that may be 
less ambiguous as indicators of survey quality. 

We propose an indicator, which we call an R-indicator 
(‘R’ for representativeness), for the similarity between the 
response to a survey and the sample or the population under 
investigation. This similarity can be referred to as “repre- 
sentative response’’. In the literature, there are many differ- 
ent interpretations of the ‘representativeness’ concept. See 
Kruskal and Mosteller (1979a, b and c) for a thorough 
investigation of the statistical and non-statistical literature. 
Rubin (1976) introduced the concept of ignorable non- 
response; the minimal conditions that allow for unbiased 
estimation of a statistic. Some authors explicitly define 
representativeness. Hajek (1981) links “representative” to 
the estimation of population parameters; the pair formed by 
an estimator and a missing-data mechanism are repre- 
sentative when, with probability one, the estimator is equal 
to the population parameter. Following Hajek’s definition, 
calibration estimators (e.g., Samdal, Swensson and Wretman 
2003) are representative for the auxiliary variables that are 
calibrated. Bertino (2006) defines a so-called univariate 
representativeness index for continuous random variables. 
This index is a distribution-free measure based on the 
Cramér — Von Mises statistic. Kohler (2007) defines what he 
calls an internal criterion for representativeness. His 


univariate criterion resembles the Z-statistic for population 
means. 

We separate the concept of representativeness from the 
estimation of a specific population parameter but relate this 
concept to the impact on the overall composition of 
response. By separating indicators from a specific para- 
meter, they can be used as tools for comparing different 
surveys and surveys over time, and for a comparison of 
different data collection strategies and modes. Also, the 
measure gives a multivariate perspective of the dissimilarity 
between sample and response. 

The R-indicator that we propose employs estimated 
response probabilities. The estimation of response 
probabilities implies that the R-indicator itself is a random 
variable, and, consequently, has a precision and possibly a 
bias. The sample size of a survey, therefore, plays an 
important role in the assessment of the R-indicator as we 
will show. However, this dependence exists for any 
measure; small surveys simply do not allow for strong 
conclusions about the missing-data mechanism. 

We show that the proposed R-indicator relates to 
Cramér’s V measure for the association between response 
and auxiliary variables. In fact, we view the R-indicator as a 
lack-of-association measure. The weaker the association the 
better, as this implies there is no evidence that non-response 
has affected the composition of the observed data. 

In order to be able to use R-indicators as tools for 
monitoring and comparing survey quality in the future, they 
need to have the features of a measure. That is, we want an 
R-indicator to be interpretable, measurable, able to be 
normalized and also to satisfy the mathematical properties of 
a measure. Especially since the interpretation and normal- 
ization are not straightforward features. 
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We apply the R-indicator to two studies that were 
conducted at Statistics Netherlands in 2005 and 2006. The 
objectives of those studies were the comparison of different 
data collection strategies. The studies involved different data 
collection modes and different non-response follow-up 
strategies. For each of the studies, a detailed analysis was 
done and documented. These studies are, therefore, suited to 
an empirical validation of the R-indicator. We compare the 
values of the R-indicator to the conclusions in the analyses. 
We refer to Schouten and Cobben (2007) and Cobben and 
Schouten (2007) for more illustrations and empirical 
investigations. 

In section 2, we start with a discussion of the concept of 
representative response. Next, in section 3, we define the 
mathematical notation for our R-indicator. Section 4 is 
devoted to the features of the R-indicator. Section 5 
describes the application of the R-indicator to the field 
studies. Finally, section 6 contains a discussion. 


2. The concept of representative response 


We, first, discuss what it means when a survey respon- 
dent pool is representative of the sample. Next, we make the 
concept of representativeness mathematically ngorous by 
giving it a definition. 


2.1 What does representative mean? 


Literature warns us not to single-mindedly focus on 
response rates as an indicator of survey quality. This can 
easily be illustrated by an example from the 1998 Dutch 
survey POLS (short for Permanent Onderzoek Leefsituatie 
or Integrated Survey on Household Living Conditions in 
English). 

Table 1 contains the one- and two-month POLS survey 
estimates for the proportion of the Dutch population that 
receives a form of social allowance and the proportion that 
has at least one parent that was born outside the Netherlands. 
Both variables are taken from registry data and are 
artificially treated as survey items by deleting their values 
for non-respondents The sample proportions are also given 
in Table 1. After one month, the response rate was 47.2%, 
while after the full two-month interview period, the rate was 
59.7%. In the 1998 POLS, the first month was CAPI 
(Computer Assisted Personal Interview). Non-respondents 
after the first month were allocated to CATI (Computer 
Assisted Telephone Interview) when they had a listed, land- 
line phone. Otherwise, they were allocated once more to 
CAPI. Hence, the second interview month gave another 
12.5% of response. However, from table 1 we can see that 
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after the second month, the survey estimates have a larger 
bias than after the first month. 


Table 1 
Response means in POLS for the first month of interviews and 
the full two-month interview period 


Variable After After Sample 
1 month 2 months 

Receiving social allowance 10.5% 10.4% 12.1% 

Non-native 12.9% 12.5% 15.0% 

Response rate 47.2% 59.7% 100% 


From the example, it seems clear that the increased effort 
led to a less representative response with respect to both 
auxiliary variables. But what do we mean by representative 
in general? 

It turns out that the term “representative” is often used 
with hesitation in the statistical literature. Kruskal and 
Mosteller (1979a, b and c) make an extensive inventory of 
the use of the word “representative” in the literature and 
identify nine interpretations. A number of interpretations 
they have found are omnipresent in the statistical literature. 
The statistical interpretations that Kruskal and Mosteller 
named ‘absence of selective forces’, ‘miniature of the 
population’, and ‘typical or ideal cases’ relate to probability 
sampling, quota sampling and purposive sampling. In the 
next section, we will propose a definition that corresponds to 
the ‘absence of selective forces’ interpretation. First, we will 
explain why we make this choice. 

The concept of representative response is also closely 
related to the missing-data mechanisms Missing-Completely- 
at-Random (MCAR), Missing-at-Random (MAR) and Not- 
Missing-at-Random (NMAR) that are often referred to in the 
literature, see Little and Rubin (2002). A missing-data 
mechanism is MCAR when the probability of response does 
not depend on the survey topic of interest. The mechanism 1s 
MAR if the response probability depends on observed data 
only, which is, hence, a weaker assumption than MCAR. If 
the probability depends on missing data also, then the 
mechanism is said to be NMAR. These mechanisms, in fact, 
find their origin in model-based statistical theory. Somewhat 
loosely interpreted with respect to a survey topic, MCAR 
means that respondents are on average the same as non- 
respondents, MAR means that within known subpopulations, 
respondents are on average the same as non-respondents, and 
NMAR implies that even within subpopulations, respondents 
are different. The addition of the survey topic is essential. 
Within one questionnaire, some survey items can be MCAR, 
while other items are MAR or NMAR. Furthermore, the 
MAR assumption for one survey item holds for a particular 
stratification of the population. A different item may need a 
different stratification. 
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Given that we wish to monitor and compare the response 
to different surveys in topic or time, it is not appealing to 
define a representative response as dependent on the survey 
topic itself nor as dependent on the estimator used. We focus 
instead on the quality of data collection and not on the 
estimation. This setting leads us to compare the response 
composition to that of the sample. Clearly, the survey topics 
influence the probability that households participate in the 
survey, but the influence cannot be measured or tested and, 
hence, from our perspective, this influence cannot be the 
input for assessing response quality. We propose to judge 
the composition of response by pre-defined sets of variables 
that are observed outside of the survey and can be employed 
for each survey under investigation. We want the respondent 
selection to be as close as possible to a ‘simple random 
sample of the survey sample’, i.e., with as little relation as 
possible between response and characteristics that distin- 
guish units from each other. The latter can be interpreted as 
having selective forces which are absent in the selection of 
respondents, or as MCAR with respect to all possible survey 
variables. 


2.2 Definition of a representative response subset 


Let i = 1, 2, 3,..., N be the unit labels for the popu- 
lation. By s, we denote the 0-1-sample indicator, i.e., when 
unit 7 is sampled, it takes the value 1 and 0 otherwise. By 1, 
we denote the 0-1-response indicator for unit 7. If unit 7 is 
sampled and did respond then 7, = 1. It is 0 otherwise. The 
sample size is n. Finally, 2, denotes the first-order inclu- 
sion probability of unit i. 

The key to our definitions lies in the individual response 
propensities. Let p, be the probability that unit i responds 
when it is sampled. 

The interpretation of a response propensity is not 
straightforward by itself. We follow a model-assisted 
approach, i.e., the only randomness is in the sample and 
response indicators. A response probability is a feature of a 
labelled and identifiable unit, a biased coin that the unit 
carries in a pocket, so to speak, and is, therefore, inseparable 
from that unit. With a little effort, however, all concepts can 
be translated into a model-based context. 

First, we give a strong definition. 


Definition (strong): A response subset is representative with 
respect to the sample if the response propensities p, are the 
same for all units in the population 


D; = es) eT] py Vi, (1) 
and if the response of a unit is independent of the response 
of all other units. 


If a missing-data mechanism would satisfy the strong 
definition, then the mechanism would correspond to 
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Missing-Completely-at-Random (MCAR) with respect to all 
possible survey questions. Although the definition is ap- 
pealing, the validity of it can never be tested in practice. We 
have no replicates of the response of one single unit. We, 
therefore, also construct a weak definition that can be tested 
in practice. 


Definition (weak): A response subset is representative of a 
categorical variable X with H categories if the average 
response propensity over the categories is constant 


ch 1g 
P,=—> oe, a oem (0) Om Jeet Poe nent fe (2) 
Ni, ka 


where N,, is the population size of category h, p,, is the 
response propensity of unit k in class h and summation is 
over all units in this category. 


The weak definition corresponds to a missing-data 
mechanism that is MCAR with respect to X, as MCAR 
states that we cannot distinguish respondents from non- 
respondents based on knowledge of X. 


3. R-indicators 


In the previous section, we defined strong and weak 
representative response. Both definitions make use of 
individual response probabilities that are unknown in 
practice. First, we start with a population R-indicator. From 
there on, we base the same R-indicator on a sample and on 
estimated response propensities. 


3.1 Population R-indicators 


We first consider the hypothetical situation where the 
individual response propensities are known. Clearly, in that 
case we can even test the strong definition and we simply 
want to measure the amount of variation in the response 
propensities; the more variation, the less representative in 


the strong sense. Let p = (P,, Pz, -.-, Py) be a vector of 
response propensities, let 1 = (1,1, ..., 1)’ be the N- 
vector,,of ones, and. let p,)=.1 x p. be. the vector 


consisting of the average population propensity. 

Any distance function d in [0, 1]” would suffice in 
order to measure the deviation from a strong representative 
response by calculating d(p, p,). Note that the height of 
the overall response does not play a role. The Euclidean 
distance is a straightforward distance function. When 
applied to a distance between p and p,, this measure is 
proportional to the standard deviation of the response 
probabilities 


] 
Nw! 


Sp) a Deon s, 00s (3) 
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It is not difficult to show that 


Al os 


We want the R-indicator to take values on the interval 
[0, 1] with the value 1 being strong representativeness and 
the value 0 being the maximum deviation from strong 
representativeness. We propose the R-indicator R, which is 
defined by 


S(p) < (4) 


a 
st 


R(p) = 1 — 2S(p). (5) 


Note that the minimum value of (5) depends on the 
response rate, see figure 1. For p = 1/2, it has a minimum 
value of 0. For 6 = 0 and p = 1, clearly no variation is 
possible and the minimum value is 1. Paradoxically, the 
lower bound increases when the response rate decreases 
from 1/2 to 0. For a low response rate, there is less room 
for individual response propensities to have a large variation. 


= | | 
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Average propensity 
Figure i Minimum value of R-indicator (5) as a 


function of the average response propensity 


One may view R as a lack of association measure. When 
R(p) = 1 there is no relation between any survey item and 
the missing-data mechanism. We show that R in fact has a 
close relation to the well-known x’ -statistic that is often 
used to test independence and goodness-of-fit. 

Suppose that the response propensities are only different 
for classes h defined by a categorical variable X. Let 9, 
and f, be, respectively, the response propensity and the 
population function of class h, i.e., 

Scien caks (3) Sind acct Ap" Pe be 6b (6) 

Hence, for all i with X, = h the response propensity is 
P; = P,- 

Since the variance of the response propensities is the sum 
of the ‘between’ and ‘within’ variances over classes h, and 
the within variances are assumed to be zero, it holds that 
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S*(p) 
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The ’ -statistic measures the distance between observed 
and expected proportions. However, it is only a true distance 
function in the mathematical sense for fixed marginal 
distributions f, and p. We can apply the x’ -statistic to X 
in order to ‘measure’ the distance between the true response 
behaviour and the response behaviour that is expected when 
response is independent of X. In other words, we measure 
the deviation from weak representativeness with respect 
to X. 

We can rewrite the y’ -statistic to get 


H 
(N,P, — N, P)” 
Lee h Ph nP 
N,pP 
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An association measure that transforms the y’ -statistic to 
the [0, 1] interval, see e.g., Agresti (2002), is Cramér’s V 


2 


ee (9) 
N(min{C, R} —- 1) 


where C and R are, respectively, the number of columns 
and rows in the underlying contingency table. Crameér’s V 
attains a value 0 if observed proportions exactly match 
expected proportions and its maximum is 1. In our case, the 
denominator equals N since the response indicator has only 
two categories: response and non-response. As a conse- 
quence, (9) changes into 


[x N= . 
V = _- = a S) 
N \Npd —"p) (P) 


From (10) we can see that for large N, Cramér’s V is ap- 
proximately equal to the standard deviation of the response 
propensities standardized by the maximal standard deviation 


J p(1 — p) fora fixed average response propensity Pp. 


(10) 
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3.2 Response-based R-indicators 


In section 3.1, we assumed that we know the individual 
response propensities. Of course, in practice these 
propensities are unknown. Furthermore, in a survey, we only 
have information about the response behaviour of sample 
units. We, therefore, have to find alternatives to the 
indicators R. An obvious way to do this is to use response- 
based estimators for the individual response propensities and 
the average response propensity. 

We let 6, denote an estimator for p, which uses all or a 
subset of the available auxiliary variables. Methods that 
support such estimation are, for instance, logistic or probit 
regression models (Agresti 2002) and CHAID classification 
trees (Kass 1980). By Pp we denote the weighted sample 
average of the estimated response propensities, i.e., 


6, 2) (11) 


where we use the inclusion weights. 
We replace R by the estimators R 


Note that in (12) there are in fact two estimation steps 
based on different probability mechanisms. The response 
propensities themselves are estimated and the variation in 
the propensities is estimated. We return to the consequences 
of the two estimation steps in section 4. 


3.3. Example 


We apply the proposed R-indicators to the survey data 
from the 1998 POLS that we described in section 2.1. Recall 
that the survey was a combination of face-to-face and 
telephone interviewing in which the first month was CAPI 
only. The sample size was close to 40,000 and the response 
rate was approximately 60%. We linked the fieldwork 
administration to the sample and deduced whether each 
contact attempt resulted in a response. This way, we can 
monitor the pattern of the R-indicator during the fieldwork 
period. 

For the estimation of response rates we used a logistic 
regression model with region, ethnic background and age as 
independent variables. Region was a classification with 16 
categories, the 12 provinces and the four largest cities — 
Amsterdam, Rotterdam, The Hague and Utrecht — as 
separate categories. Ethnic background has seven categories: 
native, Moroccan, Turkish, Surinam, Dutch Antilles, other 
non-western non-native and other western non-native. The 
classification is based on the country of birth of the parents 
of the selected person. The age variable has three categories: 
0 — 34 years, 35 — 54 years, and 55 years and older. 
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In figure 2, R is plotted against the response rate for the 
first six contact attempts in POLS. The leftmost value 
corresponds to the respondent pool after one attempt was 
made. For each additional attempt, the response rate in- 
creases but the indicator shows a drop in representativeness. 
This result confirms the findings in Schouten (2004). 


0.0 OZ 0.4 0.6 0.8 1.0 
Response rate 


Figure 2 R-indicator for first six contact attempts in 
POLS 1998 


4. Features of R-indicators 


In section 3, we propose a candidate indicator for repre- 
sentativeness. However, other indicators can be constructed. 
There are many association measures or fit indexes, e.g., 
Goodman and Kruskal (1979), Bentler (1990) and Marsh, 
Balla and McDonald (1988). Association measures have a 
strong relation to R-indicators. Essentially, R-indicators 
attempt to measure in a multivariate setting the lack of 
association. In this section, we discuss the desired features of 
R-indicators. We show that the proposed R-indicator R 
allows for a straightforward upper bound on the non- 
response bias. 


4.1 Features in general 


We want R-indicators to be based on a distance function 
or metric in the mathematical sense. The triangle inequality 
property of a distance function allows for a partial ordering 
of the variation in response propensities which enables 
interpretation. A distance function can easily be derived 
from any mathematical norm. In section 3, we chose to use 
the Euclidean norm as this norm is commonly used. The 
Euclidean norm led us to an R-indicator that uses the 
standard deviation of response propensities. Other norms, 
like the supremum norm, would lead us to alternative 
distance functions. In section 4.3, however, we show that the 
Euclidean norm based R-indicators have interesting 
normalization features. 
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We must make a subtle distinction between R-indicators 
and distance functions. Distance functions are symmetric 
while an R-indicator measures a deviation with respect to a 
specific point, namely the situation where all response 
propensities are equal. If we change the vector of individual 
propensities, then this point is in most cases shifted. 
However, if we fix the average response propensity, then the 
distance function facilitates interpretation. 

Apart from a relation to a distance function, we want to 
be able to measure, interpret and normalize the R-indicators. 
In section 3.2, we already derived response-based estimators 
for ‘population’ R-indicators that are not measurable when 
response propensities are unknown and all we have is the 
response to a survey. Hence, we made R-indicators mea- 
surable by switching to estimators. The other two features 
are discussed separately in the next two sections. 


4.2 Interpretation 


The second feature of R-indicators is the ease with which 
we can interpret their values and the concept they are 
measuring. We moved to an estimator for an R-indicator that 
is based on the samples of surveys and on estimators of 
individual response probabilities. Both have far-reaching 
consequences for the interpretation and comparison of the 
R-indicator. 

Since the R-indicator is an estimator itself, it is also a 
random variable. This means that it depends on the sample, 
i.é., it is potentially biased and has a certain accuracy. But 
what is it estimating? 

Let us first assume that the sample size is arbitrarily large 
so that precision does not play a role and also suppose the 
selection of a model for response propensities is no issue. In 
other words, we are able to fit any model for any fixed set of 
auxiliary variables. 

There is a strong relation between the R-indicator and the 
availability and use of auxiliary variables. In section 2, we 
defined strong and weak representativeness. Even in the case 
where we are able to fit any model, we are not able to 
estimate response propensities beyond the ‘resolution’ of the 
available auxiliary variables. Hence, we can only draw 
conclusions about weak representativeness with respect to 
the set of auxiliary variables. This implies that whenever an 
R-indicator is used, it is necessary to complement its value 
by the set of covariates that served as a grid to estimate 
individual response propensities. If the R-indicator is used 
for comparative purposes, then those sets must be the same. 
We must add that it is not necessary for all auxiliary 
variables to be used for the estimation of propensities, since 
they may not add any explanatory power to the model. 
However, the same sets should be available. The R-indicator 
then measures a deviation from weak representativeness. 
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The R-indicator does not capture differences in response 
probabilities within subgroups of the population other than 
the subgroups defined by the classes of X. If we let 
h =1, 2, ..., H again denote strata defined by X, N, be 
the size of stratum h, and fp, be the population average of 
the response probabilities in stratum h, then it is not 
difficult to show that R is a consistent estimator of 


H 
Ry (p) = 1-2) Ns. - BY, (13) 
Mn h=1 


when standard models like logistic regression or linear 
regression are used to estimate the response probabilities. Of 
course, (13) and (5) may be different. 

In practice, the sample size is not arbitrarily large. The 
sample size affects both estimation steps; the estimation of 
response propensities and the estimation of the R-indicator 
using a sample. 

If we knew the individual response propensities, then the 
sample-based estimation of the R-indicator would only lead 
to variance and not to bias. We would be able to estimate the 
population R-indicator without bias. Hence, for small 
sample sizes, the estimators would have a small precision 
which could be accounted for by using confidence intervals 
instead of merely point estimators. 

The implications for the estimation of response proba- 
bilities are, however, different because of model selection 
and model fit. There are two alternatives. Either one imposes 
a model to estimate propensities fixing the covariates 
beforehand, or one lets the model be dependent on the 
significant contribution of covariates with respect to some 
predefined level. In the first case, again no bias is introduced 
but the standard error may be affected by over fitting. In the 
second case, the model for the estimation of response 
propensities depends on the size of the sample; the larger the 
sample, the more interactions that are accepted as sig- 
nificant. Although it is standard statistical practice to fit 
models based on a significance level, model selection may 
introduce bias and variance to the estimation of any R- 
indicator. This can be easily understood by going to the 
extreme of a sample of, say, size 10. For such a small 
sample, no interaction between response behaviour and 
auxiliary characteristics will be accepted, leaving an empty 
model and an estimated R-indicator of 1. Small samples 
simply do not allow for the estimation of response 
propensities. In general, a smaller sample size will, thus, 
lead to a more optimistic view on representativeness. 

We should make a further subtle distinction. It is possible 
that, for one survey, a lot of interactions contribute to the 
prediction of response propensities but each one contributes 
very little, while in another survey there is only one but it 
strongly contributes a single interaction. None of the small 
contributions may be significant, but together they are as 
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strong as the one large contribution that is significant. 
Hence, we would be more optimistic in the first example 
even if sample sizes would be comparable. 

These observations show that one should always use an 
R-indicator with some care. It cannot be viewed as separate 
from the auxiliary variables that were used to compute it. 
Furthermore, the sample size has an impact on both bias and 
precision. 


4.3. Normalization 


The third important feature is the normalization of an R- 
indicator. We want to be able to attach bounds to an R- 
indicator so that the scale of an R-indicator, and, hence, 
changes in the R-indicator get a meaning. Clearly, the 
interpretation issues that we raised in the previous section 
also affect the normalization of the R-indicator. Therefore, in 
this section we assume the ideal situation where we can 
estimate response propensities without bias. This assumption 
holds for large surveys. We discuss the normalization of the 
R-indicator R. 


4.3.1 Maximal absolute bias and maximal root mean 
square error 


We show that for any survey item Y, the R-indicator can 
be used to set upper bounds to the non-response bias and to 
the root mean square error (RMSE) of adjusted response 
means. We use these bounds of the R-indicator to show the 
impact under worst-case scenarios. 

Let Y be some variable that is measured in a survey and 
let ,,; be the Horvitz-Thompson estimator for the popu- 
lation mean based on the survey response. It can be shown 
(e.g., Bethlehem 1988, Sarmdal and Lundstrém 2005) that its 
bias B(),,,) is approximately equal to 


ivepmypia) (14) 


with C(y, p) =1/N Y2(y, — ¥) (P; — P) the population 
covariance between the survey items and the response 
probabilities. For a close approximation of the variance 
S°(Vur) Of Vy we refer to Bethlehem (1988). 

A normalization of R is found by the Cauchy-Schwarz 
inequality. This inequality states that the covariance between 
any two variables is bounded in absolute sense by the 
product of the standard deviations of the two variables. We 
can translate this to bounds for the bias (14) of Vive 


Hany AOS) 2s ai MODS”) 
¥ p 2p 


= B,,(P, Y). (15) 
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Clearly, we do not know the upper bound B(p, y) in 
(15) but we can estimate it using the sample and the 
estimated response probabilities. We denote the estimator by 
BAG, 7y): 

In a similar way, we can set a bound to the root mean 
square error (RMSE) of 4 It holds approximately that 


RMSE (Yy7) + 9 Be Sexes) ir s* (Yur) 
< (Bie, y) + 5° Our) 


= E,,(P, Y). (16) 


Again, we do not know £)(p, y). Instead, we use the 
sample-based estimator that employs the estimated response 
probabilities, denoted by E,, (f, y). 

The bounds B (p, y) and E (f, y) are different for 
each survey item y. For comparison purposes it is, 
therefore, convenient to define a hypothetical survey item. 
We suppose that S(y) = 0.5. The corresponding bounds 
we denote by B (6) and E. (p). They are equal to 

(1 — R(p)) 


BOL gumerrgae (17) 


E,, (p) = B26) + 8 Gur)- (18) 


We compute (17) and (18) in all studies described in 
section 5. We have to note that (17) and (18) are again 
random variables that have a certain precision and that are 
potentially biased. 


4.3.2 Response-representativeness functions 


In the previous section, we used the R-indicator to set 
upper bounds to the non-response bias and to the root mean 
square error of the (adjusted) response mean. Conversely, 
we may set a lower bound to the R-indicator by demanding 
that either the absolute non-response bias or the root mean 
square error is smaller than some prescribed value. Such a 
lower bound may be chosen as one of the ingredients of 
quality restrictions put upon the survey data by a user of the 
survey. If a user does not want the non-response bias or root 
mean square to exceed a certain value, then the R-indicator 
must be bigger than the corresponding bound. 

Clearly, lower bounds to the R-indicator depend on the 
survey item. Therefore, again we restrict ourselves a 
hypothetical survey item for which S(y) = 0.5. 

It is not difficult to show from (17) that if we demand that 


A 


Bepyeany, (19) 
then it must hold that 
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R21 - 4py = 7, ). (20) 
Analogously, using (18) and demanding that 
E,,(6) < ¥, (21) 


we alrive at 


Rizal ABA yer A Sand aCe Ait) 222) 


In (20) and (22) we let 7(y, 6) and 7,(y, fp) denote 
lower limits to the R-indicator. In the following section, we 
refer to 7,(y, 6) and 7,(y, fp) as response-representativeness 
functions. We compute them for the studies in section 5. 


4.3.3 Example 


We again illustrate the normalization with the same 
example used in sections 2.1 and 3.3. Figure 3 contains the 
response-representativeness function 7(y, 6) and the 
observed R-indicators R for the six contact attempts in 
POLS 1998. Three values of y are chosen, y = 0.1; 
y = 0.075 and y = 0.05. 


S 
_ i ee 
ova) WE a oe ee 
Oo ia, iia. = Sd Saas 
aot, Whe nikitos eel 
oO | a wk 
ow oO 
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co) 
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So 
ae 
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Response rate 


Figure 3 Lower bounds for R-indicator R for the first 
six contact attempts of POLS 1998. Lower 
bounds are based on y = 0.1, y = 0.075 and 
Y = 0.05 


Figure 3 indicates that after the second contact attempt, 
the values of the R-indicator exceed the lower bound 
corresponding to the 10%-level. After four attempts, the R- 
indicator is close to the 7.5%-level. However, the values 
never exceed the other lower bound that is based on the 5%- 
level. 

In figure 4, the maximal absolute bias B. (6) is plotted 
against the response rate of the six contact attempts. After 
the third contact attempt, the R-indicator has converged on a 
value around 8%. 
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Figure 4 Maximal absolute bias for the first six contact 
attempts of POLS 1998 


5. Application of the R-indicator 


In this section, we apply the R-indicator to two studies 
that investigate different non-response follow-up strategies 
and different combinations of data collection modes. The 
first study involves the Dutch Labour Force Survey (LFS). 
The study is an investigation of both the call-back approach 
(Hansen and Hurwitz 1946) and the basic-question approach 
(Kersten and Bethlehem 1984). The second study deals with 
mixed-mode data collection designs applied to the Dutch 
Safety Monitor survey. 

In sections 5.2 and 5.3 we take a closer look at the studies 
in connection with the representativeness of their different 
fieldwork strategies. First, in section 5.1 we describe how 
we approximate standard errors. 


5.1 Standard error and confidence interval 


If we want to compare the values of the R-indicator for 
different surveys or data collection strategies, we need to 
estimate their standard errors. 

The R-indicator R involves the sample standard 
deviation of the estimated response probabilities. This means 
that there are two random processes involved. The first 
process is the sampling of the population. The second 
process is the response mechanism of the sampled units. If 
the true response probabilities were known, then drawing a 
sample would still introduce uncertainty about the popu- 
lation R-indicator and, hence, lead to a certain loss of 
precision. However, since we do not know the true response 
probabilities, these probabilities are estimated using the 
sample. This introduces additional precision loss. 

An analytical derivation of the standard error of R is not 
straightforward due to the estimation of the response prob- 
abilities. In this paper, we are resigned to naive numerical 
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approximations of the standard error. We estimate the 
standard error of the R-indicator by non-parametric boot- 
strapping (Efron and Tibshirani 1993). The non-parametric 
bootstrap estimates the standard error of the R-indicator by 
drawing a number 5 = 1, 2, ..., B of so-called bootstrap 
samples. These are samples drawn independently and with 
replacement from the original dataset, of the same size n as 
the original dataset. The R-indicator is calculated for every 
bootstrap sample 6. We thus obtain B replications of the R- 
indicator; Res b =1, 2, ..., B. The standard error for 
the empirical distribution of these B replications is an 
estimate for the standard error of the R-indicator, that is 


1 B A a 
I ya. (Re " RT) 


pad oe 


where R®’ = 1/B peep ical is the average estimated R- 
indicator. 

In the approximations, we take B = 200 for all studies. 
We experimented with larger numbers of B of up to 
B = 500, but found that in all cases, the estimate of the 
standard error had converged by B = 200. 

We determine 100(1 — a)% confidence intervals by 
assuming a normal approximation of the distribution of R 
employing the estimated standard errors using (23) 


Cie is (Reena scisb?) (24) 


with <a the 1 -— a quantile of the standard normal 
distribution. 


5.2 Labour Force Survey; follow-up study 2005 


From July to December 2005, Statistics Netherlands 
conducted a large-scale follow-up of non-respondents in the 
Dutch Labour Force Survey (LFS). In the study, two 
samples of non-respondents in the LFS were approached 
once more using either a call-back approach (Hansen and 
Hurwitz 1946) or a basic-question approach (Kersten and 
Bethlehem 1984). The samples consisted of LFS households 
that refused, were not processed or were not contacted in the 
LFS for the months July — October. In the design of the 
follow-up study, we used the recommendations in the 
studies by Stoop (2005) and Voogt (2004). 

The main characteristics of the call-back and basic- 
question approaches applied to the LFS are given in Table 2. 
For more details, we refer to Schouten (2007) and Cobben 
and Schouten (2007). The call-back approach employed the 
original household questionnaire in CAPI, while the basic- 
question approach used short questionnaires in a mixed- 
mode setting. The mixed-mode design involved web, paper 
and CATI. CATI was used for all households with a listed 
phone number. Households without a listed phone number 
received an advance letter, a paper questionnaire and a login 
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to a secure website containing the web questionnaire. 
Respondents were left the choice to fill in either the paper or 
web questionnaire. 


Table 2 
Characteristics of the two approaches in the follow-up study 


Call-back approach Basic-question approach 


LFS questionnaire to be 
answered by all members of 
the household in CAPI 


e A strongly condensed 
questionnaire with key 
questions of the LFS which 
takes between | and 3 
minutes to answer or fill in 


28 interviewers 
geographically selected from 
historically best-performing e Mixed-mode data collection 
interviewers design using web, paper 
and CATI 


Interviewer was different 
from interviewer that 
received non-response 


The questionnaire was to be 
answered by one person per 
household following the 


Interviewers received ‘ 
next birthday method 


additional training in 
doorstep interaction 


The timing is one week 
after the household is 
processed as a non-response 


Extended fieldwork period 
of two months 


Interviewer could offer 
incentives 


Interviewers could receive a 
bonus 


A paper summary of the 
characteristics of the non- 
responding household was 
sent to the interviewer 


Allocation of address one 
week after non-response 


The sample size of the LFS pilot was n=18,074 
households, of which 11,275 households responded. The 
non-responding households were stratified according to the 
cause of non-response. Households that were not processed 
or contacted, and households that refused were eligible for a 
follow-up. It was considered to be unethical to follow-up 
households that did not respond due to other causes like 
illness. In total, 6,171 households were eligible. From these 
households, two simple random samples were drawn of size 
775. In the analyses, the non-sampled eligible households 
were left out. The sampled eligible households received a 
weight accordingly. The 11,275 LFS respondents and the 
628 ineligible households all received a weight of one. This 
implies that the inclusion probabilities are unequal for this 
example. 

Schouten (2007) compared the LFS respondents to the 
converted and persistent non-respondents in the call-back 
approach using a large set of demographic and socio- 
economic characteristics. He used logistic regression models 
to predict the type of response. He concluded that the 
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converted non-respondents in the call-back approach are 
different from the LFS respondents with respect to the 
selected auxiliary variables. Furthermore, he found no 
evidence that the converted non-respondents were different 
from persistent non-respondents with respect to the same 
characteristics. These findings have led to the conclusion 
that the combined response of the LFS and call-back 
approach is more representative with respect to the selected 
auxiliary variables. 

The additional response in the basic question approach 
was analyzed by Cobben and Schouten (2007) using the 
same set of auxiliary variables and employing the same 
logistic regression models. For this follow-up, the findings 
were different for households with and without a listed 
phone number. When restricted to listed households, they 
found the same results as for the call-back approach; the 
response becomes more representative after the addition of 
the listed basic-question respondents. However, for the 
overall population, i.e., including the unlisted households, 
the inverse was found. The basic-question approach gives 
‘more of the same’ and, hence, sharpens the contrast 
between respondents and non-respondents. Combining LFS 
response with basic-question response leads to a less 
representative composition. In the logistic regression models 
by Cobben and Schouten (2007) the 0-1 indicators for 
having a listed phone number and having a paid job gave a 
significant contribution. 

Cobben and Schouten (2007) and Schouten (2007) used 
the set of auxiliary variables listed in Table 3. The auxiliary 
variables were linked to the sample from various registers 
and administrative data. The variables in logistic regression 
models for response probabilities were selected when the 
variables gave a significant contribution at the 5% level. 
Otherwise, they were excluded. 


Table 3 

The auxiliary variables in the studies by Schouten (2007) and 
Cobben and Schouten (2007). The household core is the head of 
the household and his or her partner if present 


Variable 

Household has a listed phone number 

Region of the country in 4 classes 

Province and 4 largest cities 

Average age in 6 classes 

Ethnic group in 4 classes 

Degree of urbanization in 5 classes 

Household type in 6 classes 

Gender 

Average house value at zip code level in 11 classes 

At least one member of household core is self-employed 

At least one member of household core has a subscription to the CWI 
At least one member of household core receives social allowance 
At least one member of household core has a paid job 


At least one member of household core receives disability allowance 
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Table 4 shows the weighted sample size, response rate, 
R, CIpts, B, and E, for the response to the LFS, the 
response of the LFS combined with the call-back response 
and the response of the LFS combined with the basic- 
question response. The standard errors are relatively large 
with respect to the studies in subsequent sections due to the 
weighting. There is an increase in R when the call-back 
respondents are added to the LFS respondents. As both the 
response rate and the R-indicator increase, the maximal 
absolute bias B decreases. The confidence intervals CI>5, 
for the LFS response and the combined LFS and call-back 
response overlap. However, the one-sided null hypothesis 


Hy: Rurs — Rupsecp 2 0 is rejected at the 5%-level. 


Table 4 

Weighted sample size, response rate, R-indicator, confidence 
interval, maximal bias and maximal RMSE for LFS, LFS plus call- 
back, and LFS plus basic-question for the extended set of auxiliary 
variables 


Response n Rate R cles 05 Bn E es 
LFS 18,074 62.2% 80.1% (77.5-82.7) 8.0% 8.0% 
LFS + call-back 18,074 76.9% 85.1% (82.4-87.8) 48% 4.9% 


LFS + basic-question _ 18,074 75.6% 78.0% _(75.6-80.4) 7.3% 7.3% 


In Table 4, there is a decrease in R when we compare 
the LFS response to the combined response with the basic- 
question approach. This decrease is not significant. B,, 
slightly decreases. In Table 5, this comparison is restricted to 
households with a listed phone number. The R-indicator in 
general is much higher than for all the households. Because 
the sample size is now smaller, the estimated standard errors 
are larger as is reflected in the width of the confidence 
interval. Be is decreased. For the combined response in the 
LFS and the basic-question approach, we see an increase of 
R but again this increase is not significant. B. decreases. 


Table 5 

Sample size, response rate, R-indicator, confidence interval, 
maximal bias and maximal RMSE for LFS, and LFS plus 
basic-question restricted to households with listed phone 
numbers and for the extended set of auxiliary variables 


Response n Rate R Cire 05 Bm Em 
BES 10,135 68.5% 86.3% (83.1-89.5) 5.0% 5.1% 
LFS + basic-question _10,135 83.0% 87.5% (84.3-90.7) 3.8% 3.8% 


We find in the example of the LFS follow-up that the R- 
indicators confirm the conclusions for the call-back ap- 
proach and the basic question approach. Furthermore, the 
increase in the R-indicator that follows by adding the call- 
back response is significant at the 5% level. 
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5.3 Safety Monitor; pilot mixed-mode 2006 


In 2006, Statistics Netherlands conducted a pilot on the 
Safety Monitor to investigate mixed-mode data collection 
strategies. See Cobben, Janssen, Berkel and Brakel (2007) 
for details. The regular Safety Monitor surveys individuals 
of 15 years and older in the Netherlands about issues that 
relate to safety and police performance. The Safety Monitor 
is a mixed-mode survey. Persons with a listed phone number 
are approached by CATI. Persons that cannot be reached by 
telephone are approached by CAPI. In the 2006 pilot, the 
possibility of using the Internet as one of the modes in a 
mixed-mode strategy was evaluated. Persons in the pilot 
were first approached with a web survey. Non-respondents 
to the web survey were re-approached by CATI when they 
had a listed phone number and by CAPI otherwise. In Table 
6 we give the response rates for the normal survey, the pilot 
response to the web only, and the response to the pilot as a 
whole. The response to the web survey alone is low. Only 
30% of the persons filled in the web questionnaire. This 
implied that close to 70% of the sampled units were re- 
allocated to either CAPI or CATI. This resulted in an 
additional response of approximately 35%. The overall 
response rate is slightly lower than that of the normal survey. 

Fouwels, Janssen and Wetzels (2006) performed a 
univariate analysis of response compositions. They argue 
that the response rate is lower for the pilot but that this 
decrease is quite stable over various demographic sub- 
groups. They observe a univariate decline in response rate 
for the auxiliary variables age, ethnic group, degree of 
urbanization and type of household. However, they do find 
indications that the response becomes less representative 
when the comparison is restricted to the web respondents 
only. This holds, not surprisingly, especially for the age of 
the sampled persons. 

Table 6 contains the sample size, response rate, R, 
(Giles B and By for three groups: the regular survey, the 
pilot survey restricted to web and the pilot survey as a 
whole. The auxiliary variables age, ethnic group, degree of 
urbanization and type of household were linked from 
registers and were selected in the logistic model for the 
response probabilities. Table 6 shows that the R-indicator for 
the web response is lower than that of the regular survey. 
The corresponding p -value is close to 5%. As a cones- 
quence of both a low response rate and a low R-indicator, 
the maximal absolute bias B, is more than twice as high as 
for the regular survey. However, for the pilot as a whole, 
both the R-indicator and B_ are close to the values of the 
regular survey. Due to the smaller sample size of the pilot, 
the estimated standard errors are larger than in the regular 
survey. 
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Table 6 

Sample size, response rate, R-indicator, confidence interval, 
maximal bias and maximal RMSE for response for the regular 
Safety Monitor, the pilot with web only and the pilot with web 
and CAPI/CATI follow-up 


Response n Rate R Clea. fo E, 
Regular 30,139 68.9% 81.4%  (80.3-82.4) 68% 68% 
Pilot - web 3,615 30.2% 77.8% (75.1-80.5) 18.3% 18.4% 


Pilot- web plus 3,615 64.7% 81.2% _(78.3-84.0) 73% 74% 


The findings in Table 6 do not contradict those of 
Fouwels ef al. (2006). We also find that the web response in 
the pilot has a less balanced composition, whereas the 
composition of the full pilot response is not markedly worse 
than that of the Safety Monitor itself. 


6. Discussion 


We have three main objectives in this paper: a math- 
ematically mgorous definition and perception of repre- 
sentative response, the construction of a potential indicator 
for representativeness, and the empirical illustrations of the 
indicator’s use. As we saw, the proposed indicator is an 
example of what we call R-indicators, where ‘R’ stands for 
representativeness. With the empirical illustration, we want 
to find support for the idea that such R-indicators are 
valuable tools in the comparison of different surveys and 
data collection strategies. R-indicators are useful if they 
confirm findings in elaborate analyses of studies that involve 
multiple surveys in time or on a topic. 

The R-indicator in this paper is promising because it can 
easily be computed and allows for interpretation and 
normalization when response propensities can be estimated 
without error. The application to real survey data shows that 
the R-indicator confirms earlier analyses of the non-response 
composition. Other R-indicators can, of course, simply be 
constructed by choosing different distance functions be- 
tween vectors of response propensities. The R-indicator and 
graphical displays showed in this paper can be computed 
using most standard statistical software packages. 

The computation of R-indicators is sample-based and 
employs models for individual response propensities. Hence, 
R-indicators are random variables themselves and there are 
two estimation steps that influence their bias and variance. 
However, it is mostly the modelling of response propensities 
that has important implications. The restriction to the sample 
for the estimation of R-indicators implies that those 
indicators are less precise, but this restriction does not 
introduce a bias asymptotically. Model selection and model 
fit usually are performed by choosing a significance level 
and adding only those interactions to the model that give a 
significant contribution. The latter means that the size of the 
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sample and the availability of auxiliary variables play an 
important role in the estimation of response propensities. 
Bias may be introduced by the model selection strategy. 
There are various obvious approaches for dealing with the 
dependence on the size of the sample. One may not do a 
model selection but fix a stratification beforehand. That way, 
bias is avoided but standard errors are not controlled and 
may be considerable. One may also let empirical validation 
be the input to develop ‘best practices’ for R-indicators. 

We applied the proposed R-indicator to two studies that 
were conducted at Statistics Netherlands in recent years, and 
that were thoroughly investigated by other authors. The 
increase or decrease in the R-indicator conforms to the more 
detailed analyses done by these authors. We, therefore, 
conclude that R-indicators can be valuable tools. However, 
more empirical evidence is clearly needed. 

The application of the R-indicator showed that there is no 
clear relation between response rate and representativeness 
of response. Larger response rates do not necessarily lead to 
a more balanced response. Not surprisingly, we do find that 
higher response rates reduce the risk of non-response bias. 
The higher the response rate, the smaller the maximal 
absolute bias of survey items. 

Application to the selected studies showed that standard 
errors do decrease with increasing sample size as expected, 
but they are still relatively large for modest sample sizes. For 
example, for a sample size of 3,600, we found a standard 
error of approximately 1.3%. Hence, if we assume a normal 
distribution, then the 95% confidence interval has an 
approximate width of 5.4%. The sample size of the LFS is 
about 30,000 units. The standard error is approximately 


0.5% and the corresponding 95% confidence interval is - 


approximately 2% wide. The standard errors are larger than 
we expected. 

This paper contains a first empirical study of an R- 
indicator and its standard error. Much more theoretical and 
empirical research is necessary to fully understand R- 
indicators and their properties. First, we did not consider 
survey items at all. Clearly, it is imperative that we do this in 
the future. However, as we already argued, R-indicators are 
dependent on the set of auxiliary variables. It can, therefore, 
be conjectured that, as for non-response adjustment 
methods, the extent to which R-indicators predict non- 
response bias of survey items is dependent on the missing- 
data mechanism. In a missing-data mechanism that is 
strongly non-ignorable, R-indicators will not do a good job. 
However, without knowledge about the missing-data 
mechanism, no other indicator would either. For this reason, 
we constructed the notion of maximal absolute bias, as this 
gives a limit to non-response bias under the worst-case 
scenario. A second topic of future research is a theoretical 
derivation of the standard error of the R-indicator used in 
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this paper. The non-parametric bootstrap errors only give 
naive approximations. However, if we want R-indicators to 
play a more active role in the comparison of different 
strategies, then we need (approximate) closed forms. Third, 
we will need to investigate the relation between the selection 
and number of auxiliary variables and the standard errors of 
the R-indicator. 
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Stratified balanced sampling 


Guillaume Chauvet ' 


Abstract 


In the selection of a sample, a current practice is to define a sampling design stratified on subpopulations. This reduces the 
variance of the Horvitz-Thompson estimator in comparison with direct sampling if the strata are highly homogeneous with 
respect to the variable of interest. If auxiliary variables are available for each individual, sampling can be improved through 
balanced sampling within each stratum, and the Horvitz-Thompson estimator will be more precise if the auxiliary variables 
are strongly correlated with the variable of interest. However, if the sample allocation is small in some strata, balanced 
sampling will be only very approximate. In this paper, we propose a method of selecting a sample that is balanced across the 
entire population while maintaining a fixed allocation within each stratum. We show that in the important special case of 
size-2 sampling in each stratum, the precision of the Horvitz-Thompson estimator is improved if the variable of interest is 
well explained by balancing variables over the entire population. An application to rotational sampling is also presented. 


Key Words: Rotational sampling; Maximum entropy; Cube method; Stratification; Unequal probability sampling. 


1. Introduction 


In the case of stratified sampling, a population U is 
partitioned into H_ subpopulations U,,h = 1,..., H 
called strata, in which samples S,,h = 1,..., H are 
selected according to independent sampling designs p,, 
h =1,..., H, respectively. The inclusion probability of 
unit k is the probability 7m, that unit & is in the sample, 
and the joint inclusion probability is the probability ,, that 
two distinct units & and / are jointly in the sample. We will 
write m=(T,),-<y and Maes (Oop) tte We assume that 
within each stratum U,, design p,(.) is of fixed size. In 
particular, then, we have ¥,-u,%,=7,, h =1, .... H, 
where n, denotes the allocation in stratum U,. In the rest 
of the paper, we assume that all sample sizes for stratum n, 
are integers. 

The Horvitz-Thompson estimator ¢,, = Y,<s Z,/™ = 
Dh fi» Where 7 =Yyes Z,/m,, provides an unbiased 
estimate of t, = Dil, 7, where t7 = Y,<y,Z, denotes the 
total of the variable (vector) z over U, . In the particular 
case where z, = y, 1s scalar, the variance of the Horvitz- 
Thompson estimator is given by the Sen-Yates-Grundy 
variance formula: 


pS a (1%) na 2 7 2. . (1) 
k 


ys h=| k#leU, 


Var(t,,.) = 


This variance is small if the strata are homogeneous with 
respect to the variable of interest, specifically if y,/m, 1s 
approximately constant within each stratum. 

If a vector x = (x, ..., x,) of g auxiliary variables is 
available prior to sample selection for each individual in the 
population, the sampling within each stratum can be 
improved with the cube algorithm (Deville and Tillé 2004), 


which selects balanced samples. Sampling design p,(.) is 
said to be balanced on the x variables if the equations 


A igen i (2) 


are exactly satisfied. The variance of the Horvitz-Thompson 
estimator is therefore zero for the estimate of the total of the 
balancing variables. In the particular case where x = 7, ie., 
if the inclusion probability is the only balancing variable, (2) 
reduces to 

Degen baie tod (3) 

keS, keU, 

Hence, stratified sampling of fixed size in each stratum is 

a particular case of balanced sampling. For any given 
number of constraints, an exactly balanced sample generally 
cannot be found. Suppose, for example, that population U, 
contains 100 individuals on whom is defined a variable x 
with two possible values, 0 and 1, and that 53 individuals in 
the population have the value 0 for that variable. Selecting a 
size-10 equal-probability sample balanced on variable x 
would mean selecting a sample containing 5,3 individuals 
for whom x =0 and 4,7 individuals for whom x =1, 
which is impossible. Consequently, the goal is generally to 
select an approximately balanced sample, such that 


paee Ve (4) 


With the cube method (Deville and Tillé 2004), we can 
select approximately balanced samples on any number of 
variables, maintaining exactly a predetermined set of 
inclusion probabilities 2. The method is composed of two 
phases: the flight phase and the landing phase. At each step 
in the flight phase, we decide at random to either select or 
permanently discard one of the population units. At the end 
of the flight phase, we have, in each stratum U,, a vector 
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E-mail: chauvet@ensai.fr. 


116 
wT = (ney, € (0, 1)" that satisfies the following 
conditions: 

Ete (5) 


pape (6) 


Gard { kiseul pi Oi o at pres Ii S 193 (7) 


where E denotes the expectation for the sampling method 
used in the flight phase. The vector n"* gives the outcome 
of the flight phase: 1, is 1 if unit k is selected, 0 if it is 
rejected, and between 0 and | only if the decision has not 
been made for unit & after the flight phase. Equations (5) 
and (6) ensure that the inclusion probabilities and balancing 
constraints are maintained perfectly at the end of the flight 
phase. Equation (7) ensures that a decision remains to be 
made for no more than q individuals in each stratum U,, 
where q is the number of balancing variables. The flight 
phase ends when the balancing constraints can no longer be 
exactly satisfied. The landing phase consists in defining, 
conditionally on the outcome of the flight phase, an optimal 
sampling design defined on the remaining population /V. 
This design is optimal in that it makes it possible to 
complete the sampling while minimizing the variance, 
conditionally on the outcome of the flight phase, of the 
Horvitz-Thompson estimator of the balancing variables . 
The remaining units are sampled, conditionally on the 
outcome of the flight phase, with inclusion probabilities 
(7, ),<y, So that the units’ unconditional inclusion probabil- 
ities (1, ),-, are maintained exactly. 

The measure of entropy associated with a sampling 
design p(.) defined on population U 1s given by 


I(p) = - >, p(s)log (p(s), 


scuU 


with the convention Olog(0) = 0. Deville and Tillé 
(2005) have shown that the balanced design with maximum 
entropy compared with other sampling designs balanced on 
the same variables and with the same inclusion probabilities 
can be regarded as the conditional of a Poisson design. 
Assuming the asymptotic normality of a multivariate 
Horvitz-Thompson estimator in the case of a Poisson 
design, they derived a variance approximation formula for 
the Horvitz-Thompson estimator for a balanced sampling 
design. In the case of stratified balanced sampling, we have 


H 
Lad b 
Varin) = > SO - BX) 8) 
h=1 keU, My 


where B, = (rev, b X,/T, x,/m) Liew, b, x,/t, y,/T). 
Deville and Tillé (2005) offer several approximations for 
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the b,. The simplest is b, = 1,(1 — 1,). The variance of 
the Horvitz-Thompson estimator will be small if, in each 
stratum, variable of interest y is well explained by balancing 
variables x. 

Sampling will be balanced in each stratum if the number 
of balancing variables remains small relative to the sample 
size. In some cases, however, the allocation to each stratum 
is too small for balanced sampling: if the stratification of the 
population is very granular, a current practice is to select a 
size-2 sample in each stratum. In that case, the only 
condition that can be imposed is a fixed sample size in each 
stratum. 

In the next section, we propose an algorithm based on the 
cube method that ensures balanced sampling across the 
entire population for selected variables and exactly main- 
tains the desired allocation within each stratum. Hence, the 
samples are no longer selected independently in each 
stratum. Precision is improved in comparison with stratified 
sampling with fixed sample size in each stratum if the 
balancing variables are strongly correlated with the variable 
of interest across the entire population. The algorithm also 
has the advantage of ensuring approximate balancing in 
each stratum, and the larger the sample size allocated to the 
stratum, the more balanced the sampling will be. 


2. Stratified balanced sampling with pooling 
of landing phases 


If sample S is selected from U in accordance with the 
stratified balanced sampling procedure described in section 
1, sampling will be balanced in each stratum as long as the 
landing phase affects a small number of individuals relative 
to the sample size. Specifically, equation (7) shows that the 
number of balancing variables must be small relative to the 
sample allocation in each stratum. In some cases, that 
constraint cannot be satisfied. The population is often 
partitioned into very small groups to make the results more 
relevant, which means decreasing the sample selected in 
each stratum; the limit generally used is a size-2 sample, 
which produces an unbiased variance estimator. 

Again, we take the case of a population U divided into 
AT strata U,, .... U%, forwhich.a yectotix, = (mz, 2,7) 
of auxiliary variables is known. We assume that the variable 
mt, 1s one of the balancing constraints, to ensure fixed-size 
sampling. Where the allocation to each stratum is too small 
for balanced sampling to apply constraints other than fixed 
size in each stratum, algorithm 1 provides an alternative 
sampling method. A flight phase is carried out inde- 
pendently in each of the H strata: we write 1" = 
(T rev,» 2=1, .... H for the probability vectors obtained 
at the end of those flight phases, 2° = (m,),-,, where V 
denotes the units that have not yet been sampled or rejected, 
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and x, = (™% eeu» > ™ ecu,» Ze ™/™%)'. The proba- 
bility vector obtained after a final flight phase over the set of 
remaining units is written =” = (7;"),-,. The set of units 
in stratum U, that have not yet been sampled or rejected at 
the end of this new flight phase is denoted V,. 


Algorithm |: Stratified balanced sampling with pooling of 
landing phases 


Step 1. Carry out a flight phase, with balancing variables 
x, and inclusion probabilities 1,, independently 
in each stratum U,. 

Carry out a flight phase, with balancing variables 
x, and inclusion probabilities 1,, on the set V of 
units remaining at the end of step 1. 

Select a fixed-size sample from each subpopulation 


W,, with inclusion probabilities 1;". 


‘Step 2. 


Step 3. 


The algorithm is based on a method used by the Institut 
National de la Statistique et des Etudes Economiques 
(INSEE) to select the primary units of the 1999 Master 
Sample. The Master Sample is a sample of dwellings 
selected in the 1999 Census for use as a sample frame for 
household surveys. A detailed description of the sampling 
design for the Master Sample is provided in Bourdalle, 
Chnistine and Wilms (2000). The dwellings are first 
grouped into urban units and rural units. In the subpopu- 
lation of units with fewer than 100,000 residents, a sample 
of about 6% is selected. We have four auxiliary variables 
(taxable net income and three age groups). The expected 
number of sample units is too small for stratified sampling 
by region, with balanced sampling on the four variables in 
each region. The regions were therefore grouped into eight 
super-regions, and the sampling processes were coor- 
dinated in such a way as to ensure both overall balanced 
sampling for the four auxiliary variables in each super- 
region and a fixed sample size in each region. 

A similar method was proposed by Rousseau and 
Tardieu (2004) for the selection of balanced samples from 
large frames using the CUBE macro available on INSEE’s 
Web site. The macro’s mun time is approximately pro- 
portional to the square of the population size. Note that 
Chauvet and Tillé (2006) proposed a fast method of 
balanced sampling whose run time depends only on the size 
of the population and which can select balanced samples 
directly from very large populations. The algorithm was 
programmed into an SAS macro (see Chauvet and Tille, 
2005) and is also available in the R Sampling Package 
prepared by Matei and Tillé (2006). In both programs, the 
second flight phase is performed by adding a constraint 
associated with each stratum to balancing variables x, and 
maintaining the fixed-size condition in each stratum. 

Using inclusion probabilities vector 2° conditionally on 
the outcome of step 1 ensures that inclusion probabilities 
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vector ™ is maintained by deconditioning from the outcome 
of step 1. At the end of step 1, equation (6) implies that 


Vh = 1...H Signin or adage india; 


keU,/0<n; <1 My keU, keU,/ny =1 


and summing these expressions yields 


y +n = dap 


key TT, keU 


ye. 


keU/n, =1 
At the end of step 2, equation (6) leads to 
X;, ee xX * 
Danes Maes, Dy Tee 
kev Ty key Ty 


and combining the last two expressions, we get 


De sa ois > a » Xi> (9) 


keU/ni =1 My keU 


which ensures that balanced sampling on the variables x, is 
maintained exactly at the end of step 2. Step 3 completes the 
sampling process while maintaining the fixed-size constraint 
within each stratum U, and can be carried out by means of 
a linear program to limit the lack of balance (see Deville and 
Tillé 2004). 

The variance can be approximated with the variance 
formula proposed by Deville and Tillé (2005), if each flight 
phase in algorithm 1 is carried out with high entropy. 
Entropy can be increased substantially by performing a 
random sort on the population prior to sampling. In this 
case, the balancing variables are both the z, variables and 
the variables given by the product of the inclusion 
probabilities and the stratum membership indicators, which 
ensure a fixed sample size in each stratum. We have 


HM b : 
Var(f,,.) = >, =O - va) (10) 
keU Ty 
with a, = (t% yeu» ™ eeu,» 2)’ and 
r -] 
a, a) a 
i= b, —— > b, ——. 
[ Ty i l<U Tt, 1%, 
We can use the variance estimator 
x b, ers ri 
Vili vices) yh ra y a,) ( ) 


keS Tey 


proposed by Deville and Tillé (2005, page 578), with 


et 
~ b, a, a, by a Y, 
p-[p2ue) yA He 
les T, TM, T ) tes TM, M TN, 
As shown in the variance approximation formula (10), it 
is important to note that the independence of the samples 
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from the various strata is lost with the proposed stratified 
balanced sampling method. The samples from strata 
U,, .... U,, are coordinated to ensure overall balance across 
the whole population, which strips them of their inde- 
pendence. The Horvitz-Thompson estimator ie of total ¢,, 
remains unbiased. Its approximate variance is derived from 
equation (10) by replacing y, with y, 1,.y,, and is given 
by 


: b . 
Vari ye > EO lev, — ("Yay (12) 


keU Ty 


with a, = (% lrey> - 


,\-1 
= [Sa sea Dy b at. 
1eU, 


™, Ty) 


‘nt 
» M |pey,» 2) and 


In the particular case where the inference does not apply 
to the entire population but to a domain D that consists of a 
small number of strata, balanced sampling overall on the z 
variables will be of little benefit. The variance of the 
Horvitz-Thompson estimator ie of total ox for variable y 
for that domain will be close to the variance for stratified 
sampling, which is given by equation (1). 


3. Quantitative results 


In this section, we carry out a brief simulation study to 
test the performance of our sampling algorithm. First, we 
generate a finite population of 1,000, partitioned into 25 
strata of equal size containing four variables: two variables 
of interest, y, and y,; and two auxiliary variables, x, and 
x,. Wariables x, and x, are generated with a gamma 
distribution with parameters 4 and 25. Variable y, is 
generated within stratum U, using the model 


VY, wi Oy t &. (13) 


The ¢, are generated with a normal distribution with 
mean 0 and variance o;. The model used to generate the 
values of y, is given by (13), with a,, = 20 and variance 
co, selected to produce a coefficient of determination R? 
approximately equal to 0.60 in each stratum. Variable y, is 
generated with the model 


Y, =O, +B, x + Y2 x) +7. (14) 


The 1 are generated with a normal distribution with 
mean 0 and variance p*. The model used to generate the 
values of y, is given by (14), with a, =500, B, = y, =5, 
and variance p’ selected to produce a coefficient of 
determination R* approximately equal to 0,60. 

We are interested in estimating the total of variables y, 
and y,. We select a sample of n = 25 (n =S0 respectively) 
units with equal probabilities using three sampling designs: 
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Design 1: Stratified simple random sampling in each 
stratum 

Design 2: Sampling balanced on variables 1, x, and x, 

Design 3: Stratified sampling balanced on _ variables 
m, x, and x,, with pooling of the landing 
phases 


In the case of stratified sampling, we have an allocation 
of size 1 (2 respectively) in each stratum. In the balanced 
designs, each flight phase is preceded by a random sort of 
the population. The variance associated with design 1 is 
calculated directly. The variance associated with designs 2 
and 3 is approximated on the basis of 10,000 simulations. 
The results are presented in Table 1. 


Table 1 

Variance associated with the estimate of the total of two 
variables for a stratified design, a balanced design and a 
stratified balanced design with pooling of landing phases 


n=25 n=50 
Total var. Totalvar. Totalvar. Total var. 
Ji J2 Yi J2 
Method  (x10°) («10° ) (x10° ) (x10? ) 
Design 1 6.05 Jas 2.95 3.48 
Design 2 14.31 3.05 7.02 1.40 
Design 3 6.00 3.63 2.98 1.54 


In each case, the proposed sampling design is comparable 
with the better of the two strategies. If the variable of interest 
is approximately constant across all strata, the proposed 
algorithm produces the same results as the stratified design. 
If the balancing variables are highly explanatory, the results 
produced by our algorithm and by direct balanced sampling 
are equivalent. The slight loss of precision comes from the 
landing phase: in the case of direct balanced sampling, we 
attempt to complete the sampling while limiting the lack of 
balance. With the proposed algorithm, the selected solution 
is suboptimal because we are imposing the additional 
constraint of a fixed size in each stratum. 

In the case of stratified balanced sampling with pooling 
of the landing phases, Table 2 shows the variance given by 
10,000 simulations and the variance given by the approxi- 
mation formula (10). 


Table 2 

Comparison of the variance given by 10,000 simulations and 
the variance given by the approximation formula in the case of 
the estimation of two totals for a stratified balanced sampling 
design with pooling of landing phases 


n=25 n=50 
Total y, Total y, Total y, Total y, 
(x108) = (x10?) = (x10) (x10") 
Simulation var. 6.0 3.6 3.0 LG 
Approximated var. aye 7a O82 bs 


Survey Methodology, June 2009 


The approximation formula proposed by Deville and 
Tillé (2005) is close to exact if the variance associated with 
the landing phase is small relative to the variance associated 
with the flight phase. In the case of the y, variable, the 
balancing variables are highly explanatory. The variance is 
therefore larger for the landing phase than for the flight 
phase, and the approximation formula understates the actual 
variance. The variance associated with the landing phase 
will be considered in future studies. 


Acknowledgements 


The author is grateful to the referees and an associate 
editor for their constructive comments and suggestions. 


119 
References 


Bourdalle, G., Christine, M. and Wilms, L. (2000). Echantillons 
maitre et emploi. Série INSEE Meéthodes, Paris, France, 21, 139- 
WEY 


Chauvet, G., and Tillé, Y. (2005). New SAS macros for balanced 
sampling. INSEE, Journées de Méthodologie Statistique, Paris. 


Chauvet, G., and Tillé, Y. (2006). A fast algorithm of balanced 
sampling. Computational Statistics, 21, 53-61. 


Deville, J.-C., and Tillé, Y. (2004). Efficient balanced sampling: The 
cube method. Biometrika, 91, 893-912. 


Deville, J.-C., and Tillé, Y. (2005). Variance approximation under 
balanced sampling. Journal of Statistical Planning and Inference, 
128, 569-591. 


Matei, A., and Tillé, Y. (2006). The R ‘sampling’ package. European 
Conference on Quality in Survey Statistics, Cardiff. 


Rousseau, S., and Tardieu, F. (2004). La macro SAS CUBE 


d échantillonnage équilibré - Documentation de _ Tl utilisateur. 
Technical report, INSEE, France. 


Statistics Canada, Catalogue No. 12-001-X 


bo “a peel cr “orditiody bot A Riana © 

fee ra RA eee ayn 
Ae Lm. Po RQt MNES 
Llorvetz= otatdint cater & 


for that ee, Ls 


7 ioe ff 


cf : MRR aN a = 


bag dE epee 


ee cra 


oth Ge. clune be the’ shrines: far aie 


siping, wi my even by eqvesicat (1) 
4, “Guantiodine puoi 
he dite aeericeh Ae taarewe cat ite darian Ney 
wt toa? eT cae ane iter “eet ‘SHU | eit ww ca, 
ienetsie a oat of 5.000, parttioned GO ue 
4 nm Ogu oe CAPTAIN I VE Hew: Vik Gavhar 
ol ries ef vied wo aiailury Capeeles, a, Dest 
Vara ut = F week whi Vn 
“ai sertruptiont { Nore cts Wik’ ¢ ey S 
t) by. nm? tel arm hi pl ree 
y, / i ‘ 
1 f, : eth oT A 
m4) @ et ) 7 4 * ifs ° by. iw 
8 ivem iy ¢1 NW owed Wo 
; cleat ae ea : Ag yank 
aL wa vie t Lag i fen. mihi 4 
“iw ~ tes 
. ice omter| Wyeth 3 oat pay anor iW 
> a ay" ¢ ‘ Flee frais watt Uc 
i ty . Wi © , 
sti nodes tk reacts icin i 
tsi Le ala oi, TD WLS 
a t ra? “i im <% “ul ' rite ude 
f f 5 : 4 eetivoly) 
j + A eu ens BL ; A Sei & Aas 


Jyhttrs! Of + ahaa eon eee 


asim 4 


V eriacee 


vartewies For’ Newt aoa 
stented nen pring 
or Sn 


me i my 


ae 
Tai vat. 


> 7? 
Weal ath? asthe Maen 
ixmien! O05 2B 
Tie py i4 3} 305 
ure OF. §.00 i 


ee 


in earch >" , the prt 1* Se coping design 1.0 UAT 

8% iv” _ @ 

VN eae tier af Tye few) en aenpeecd. ir the vartilble ot 

iy pivpatnctiately Gnint across nll strata, the f 
fies prixiuces the wap 


4 ee id : h ; van ‘ 4 recut sa ine —— 
: 7: Ceghennctny yarripies are hiebly exp! 7 
Q 


) ow algorihes and by virect alana 


, . 4 
7 wae | 


Soni The sliplt joss Of precation, Chirp: 


Lh U 
wala ghawe: ta dt cése Of doce bolanced iar 
cwrek oe Cagagh Pupils while iouting dre atk OF 
fjgeree Ve me ifs arti, We sCocted pred 
ve veliued Geemme we ar imposing. ie thd Oienad | 
joteayel Sf Vey er oy CeO KM 
‘the Qa of Ue Seed eropl eth poe \ 7 
ian ie y e-<@ * ahitews Oe Worn given by 
‘10.009 Leela es Pomiete iets hy Oa spprand: 
it iw relly g 
[etAe 2 
Conmerte@® af Ge tatedyaghae hr 16.900 simmiatives pas 


the vations gietee by (ike Aer iv'o ative Ieemete in (ee coemol 
see eyedvce daw el Od eres Tes + 001 Cty ed balanced sampling 
pegs «Wh peclitng: of te aid ehesee 


a O-@@ ee, - = 4 _ ee 
“3 awa 
futsh » Vout », Testy had ta, 
eee ee ee 
~) ete Vi a) 


serrrcepeet ia EF 


ee x9 a° 
by 26 i3 
, me 


JOURNAL OF OFFICIAL STATISTICS 
An International Review Published by Statistics Sweden 


JOS is a scholarly quarterly that specializes in statistical methodology and applications. Survey methodology and other issues 


pertinent to the production of statistics at national offices and other statistical organizations are emphasized. All manuscripts 
are rigorously reviewed by independent referees and members of the Editorial Board. 


Contents 
Volume 24, No. 4, 2008 


Assessing the Value of Bayesian Methods for Inference about Finite Population Quantities 


DOSCP IM SCULALLSR meee CMMI. tol te Sash Le Pe MONE tem erence fence er eet eRe pe Mececesd Beviecs re eenccrssteeesvnseresccnenmesvrensost 495 
Discussion 

INablanicoCOCHKCL. TmVvClOte ERaOnUnathainy se coer ee reenter eee ee ten errr: 507 
Discussion 

JB ERA (G bys toll 0) 1508 (a) Cr a he ee NUE Ue Lem Seige Aa AR Rc AP eR LL ey eae AA pec tne ae 513 


Model Averaging Methods for Weight Timming 
Mitel ReM Nicter eRe sea eart xcs Re cere ioe MECN, ce Te TEL PRON Gc dio M nas cess rowed se cahatns oder uateteesBsEFannOtHvsa cost ceseaes cuvette 3 by) 


A Note on the Asymptotic Equivalence of Jackknife and Linearization Variance Estimation for the Gini Coefficient 
NSIS Gal 81S) Fike ip ek ape peace tr a dle ad ocr Eb oto DN ei ml pn aa om ah 54] 


On Some Common Practices of Systematic Sampling 


Does a Final Coverage Check Identify and Reduce Census Coverage Errors? 
Elizabeth VMiartine DOnvA> Dilinian ss..cee. ctor eco coe eter eae oes eM Te UNC ee oc eT ea lesasay Meveievohess S71 


Privacy Concerns, Too Busy, or Just Not Interested: Using Doorstep Concerns to Predict Survey Nonresponse 
Nancy satessu ames, Dallhamiens rlcanor SING irc. ty recor er eases age rears oeeces enetnteeneeencesacats 391 


Seasonality in Investment Plans and Their Revisions 
TESS SUNG Uh a0 1 8p) poe etn ee ARG NE HN CY kT DCR a Daan ya De Di RDO Rs 612 


| EVU EL es gee O04 UP pe’ o2 1 (0) ps teeeac enh eae DER PMRIRL UeERP PCS SUB He Ov nat a ERR SOR eT J / pre SERIA ORT aE nes RE ERO 623 


Contents 
Volume 25, No. 1, 2009 


Model Averaging Methods for Weight Trimming in Generalized Linear Regression Models 
Witelaaiel RENO GCS cc viceces eects sas duces ceca sete pine ose cacenne'sas ee see ata 62s haw ne Cdn calig U eaete ate ca ] 


The Use of Sample Weights in Hot Deck Imputation 
Rebecca R: Andridge, Roderick J Cattle.) spaces oi tcc ies tec co nasa Woes -otses ssonntacrater nents tossceesotosssdedsset shoal eet eee pA 


Small Area Estimation in the Presence of Correlated Random Area Effects ~ 
Moniica Pratest: NicGla: Sabwatt sce: cccsasscscsssesetencecesnctsuesioitsosucee dou xeeeven ues eactiecob Oeame ence secu nev ae ene oT 


Nonparametric Variance Estimation for Nearest Neighbor Imputation 


On the Inter-Regional Mover Problem in Panel Household Surveys 
Takis Merkouris 22-05 s..-:00 se ahora gcre cee reteset eens Pee Bote cr Sere Meee eee rc ne 63 


Compensating for Noncoverage of Nontelephone Households in Random-Digit-Dialing Surveys: 
A Comparison of Adjustments Based on Propensity Scores and Interruptions in Telephone Service 
K-P. Srinath, Martin R: Frankel, David C. Hoaglin, Michael P. Battagiia .000.32201.2<ccxcscc-newes0-scacseceevvevecespspereeeeese ae 1) 


Population Shifts and Demographic Methods 
C; Matthew Snipp;JuamitasEamiryoli tt caso 05. oo catch cae setae anes cca rs echt are eee eee 99 


Privacy-Preserving Analysis of Vertically Partitioned Data Using Secure Matrix Products 
Alan F. Karr, Xiaodong Lin; Ashish P. Sani) Jerome P: Reiter 2..2......0.0.c.sccss:sec<see-ccascsuu-e-ssscetsceensdeieeres eet nee 125 


Using a Weighted Average of Base Period Price Indexes to Approximate a Superlative Index 
Janice Lent? Alan Fis Doriinati nce 5 ics cca css iad ae ee ee 139 


Book and Software Reviews 
Jana-Asher. Trent.D: Buskirk Jobn Hall’ Gordon Willis $.0.ccg:cp cere eee eee ee 151 


InsOthier Journals 2:55 Ss eh 2 a8 ee Ne Ta 9 ee Ne oe ee ee 165 


All inquires about submissions and subscriptions should be directed to jos@scb.se 


The Canadian Journal of Statistics La Revue Canadienne de Statistique 


CONTENTS TABLE DES MATIERES 


Volume 36, No. 3, September/septembre 2008 


Christopher A. FIELD 
IVMIOCeHIIZ DIO LOgICalCata :SeVElral VIRUCULCS 2 cne.aecds meee cree tetany te ene e arene red Reese cat Renee eae ouiade ia saase Aer Taviesedetversete 341 


Marco A.R. FERREIRA & Marc A. SUCHARD 


Bayesian analysis of elapsed times it CONntMUOUS-CMe IMaLKOV CHAINS -.1..c.45 7, <occacnecassecereuee <ctraterescureosssnvartesPeasestctsesonsensresaiterieses 355 
Amélie FILS-VILLETARD, Armelle GUILLOU & Johan SEGERS 

PIO\CCUONESUIMALOTS OF F ICKANGS:Cepencenee fUTICULONS c2ter-ccc ae UNG erin ete Ee Ete osc N ae vncavout uta de trottasactasiuvibyes 369 
Simon GUILLOTTE & Frangois PERRON 

A Bayesian estimator for the dependence function of a bivariate extreme-value distribution...........0..cccccecescessecssseeeseceeseeenseeeaees 383 
Jesus LOPEZ-FIDALGO, Raul MARTIN-MARTIN & Douglas P. WIENS 

INIA OID WATeSLEICLER: SCGUETICal L)-ODtHTIAl CESIRTIS. 5. cree a: peste ence ogee IAI hor eee eR DET Bee REE Eee eet Sere octane ree eae oa, 397 
Kerrie P. NELSON & Don EDWARDS 

On.population-based\measures of agreement Torpinary ClassiMtCauOUs cac- nna cqte se eees <a peep ga cree eee see as eevee eens ain see cas 411 
Lei NIE, Haitao CHU & Valeriy R. KOROSTYSHEVSKIY 

Bias reduction for nonparametric correlation coefficients under the bivariate normal copula 

ASSUIMDEOR WWItA KOO CELCCULOM NIRS cae lan ete sere eee Re eae eects esas scene) en asar yet esas cconeetes 427 
Jean-Fran¢gois PLANTE 

INGiparametiicradantived ikelihood: wel ghts 2scc.cy.sc5 teeters orca teas se taw eae oe ait sauce oes seca pes see nas nee tpaneonee suiadatennysues aacsera sasestastn ote oee 443 


Yichuan ZHAO & Hongkun WANG 
Empirical likelihood inference for the regression model of mean quality-adjusted lifetime with censored data............0..0.:cc 463 


Volume 36, No. 4, December/décembre 2008 


Myléne BEDARD & Jeffrey S. ROSENTHAL 
Optimal scaling of Metropolis algorithms: heading toward general target distributions ................ccccccecceeseeesceescesteeeeeeceseeessenasnes 483 


Jom DANNEMANN & Hajo HOLZMANN 
Testing for two, states in a hidden Markov model. cco... 2.20.0. <ceccecscsanssavsenvariedecng caabpe soc ehcine ras tacos sens eae cota es jee 505 


Christopher A. FIELD, Zhen PANG & Alan H. WELSH 
Bootstrapping data; with multiple levels of Variation)... j.0<:c:sqsse onczccu avers case yaeeecsvecetnesss ceascearg os uxica que nkccsupecce els owese ce eee ep 


Feifang HU, Li-Xin ZHANG, Siu H. CHEUNG & Wai S. CHAN 
Doubly adaptive biased coin designs with delayed TESpOnses :.. ss. seccceuncsscneg cwlen eevee toe race auth cok Codes wena cit cone ce eee eee 541 


Sai Man Simon KWOK & Wai Keung LI 


Onidiagnostic: checking of the autoregressive conditionaluntensity modelvatcces reste rintecees ee ree teers eee 561 
Omer OZTURK 
Inference:in:the presence of ranking error in: ranked Set SamMONny fe co coc sncsssuciercscecucnes cueae vac pen eased ae eee ee mi i 


Matias SALIBIAN-BARRERA & Ying WEI 
Weighted quantile regression with nonelliptically structured Covariates — <2, 5 fe dt i causa etneamterestcevacscceearoecues een te meet 395 


Ana-Maria STAICU & Nancy M. REID 
On: probability matching priate. oer cg rs tent at oceceatrg astccdav enc useeti ct satis oa neon gs tig avon ks ocr TR tects een ee 613 


Robin WILLINK 
Shrinkage confidence intervals for the normal mean: using a guess for greater effiCienCy .............cccccccccccesceesceescesseescessesscessenseens 623 


GUIDELINES FOR MANUSCRIPTS 


Before finalizing your text for submission, please examine a recent issue of Survey Methodology (Vol. 32, No. 2 and onward) 
as a guide and note particularly the points below. Articles must be submitted in machine-readable form, preferably in Word. 
A pdf or paper copy may be required for formulas and figures. 


ie 


it 
12 
ibe 


1.4 
Ae) 


ci 
SZ 


Layout 


Documents should be typed entirely double spaced with margins of at least 1’ inches on all sides. 

The documents should be divided into numbered sections with suitable verbal titles. 

The name (fully spelled out) and address of each author should be given as a footnote on the first page of the 
manuscript. 

Acknowledgements should appear at the end of the text. 

Any appendix should be placed after the acknowledgements but before the list of references. 


Abstract 


The manuscript should begin with an abstract consisting of one paragraph followed by three to six key words. Avoid 
mathematical expressions in the abstract. 


Style 


Avoid footnotes, abbreviations, and acronyms. 

Mathematical symbols will be italicized unless specified otherwise except for functional symbols such as “exp(-)” 
and “log(-)”, efc. 

Short formulae should be left in the text but everything in the text should fit in single spacing. Long and important 
equations should be separated from the text and numbered consecutively with arabic numerals on the right if they are 
to be referred to later. 

Write fractions in the text using a solidus. 

Distinguish between ambiguous characters, (e.g., w, @; 0, O, 0; 1, 1). 

Italics are used for emphasis. 


Figures and Tables 

All figures and tables should be numbered consecutively with arabic numerals, with titles that are as self explanatory 
as possible, at the bottom for figures and at the top for tables. 

References 


References in the text should be cited with authors’ names and the date of publication. If part of a reference is cited, 


_ indicate after the reference, e.g., Cochran (1977, page 164). 


The list of references at the end of the manuscript should be arranged alphabetically and for the same author 
chronologically. Distinguish publications of the same author in the same year by attaching a, b, c to the year of 
publication. Journal titles should not be abbreviated. Follow the same format used in recent issues. 


Short Notes 


Documents submitted for the short notes section must have a maximum of 3,000 words. 


‘S]OU YOO € SN{d Nev IIOAR JUDAIOP SAIIG SUOTJBOTUNUIWIOD Sap UOT}DES BT Inod srwNoOs syUsUINIOp Sa] 


SdAdIg SUOIBIIUNUIWIOD 


"S}U999.1 SO.QUNU 
Sd] SUPP 9SI[NN BJOPOUL 9] DIAING “SUO] NV S}LI99 3139 JUSAIOP SONASI op Saji] so] ‘UONRoTTqnd op s9uuR,| e ‘aja ‘Od 
‘q & Sale] So] JUeINOfe US s9UUL SWIQUI OUN,P 39 INIINe dQ UN, p sUOTedT{qnd sa] Jonsuysiq ‘onbisojouos1ys s1p10 
Ud a9 JUSAIOP ANaIne sWEW UN,p san say jo onbyaqeydye supsjo ud a9 Op 3X9} UN.p Ulf RI eB aIYdeISOI[Giq eT 

‘(pg oded ‘7 /6]) UeIYOOD :a]duioxy ‘sousgyoI B] Soude ayjonbe] Jonbrput ‘a9q19 ysa yFUOWNDOp UN, p o1yJed oun 
IS ‘uoneorlgnd ap ayep BJ 39 sINoyNe sap WOU J] JosI9gId JUSATOP 3}X9} 9] SULP SOLE} XNeARI] Sone ,P B SOOUSIZJOI SO] 


arydeasolqid 


‘(xneg]qe} Sap yney Ud Jo SaNSIJ sop seq ne) afqissod anb syvorydxo Issne 
3411 UN JoyOd Jd Saquie SOIJJIYD SOP IAB JINIPSUOD SIPIJO US S9JOISUINU 9.119 SNO} JUSAIOP xNeIIqQL} SO] Jo SOINSI SIT 


XnvaI qv} 39 SINS 


"SJOUI SOP INJOSSar ate} Inod sasiyyn yUos sonbiyeyt sorgjoesed So] 

(1 1£0‘O ‘0 £ © ‘mM SUID) SNSIQUI Sala}OBIBD So] JUDUIIIE[O JonSUNsIq 

‘anbiyqo aueq UN ,p Opie] B 3}X9} dg] SUBP SUOTOBIJ So] SLING 

“ULO] SN{d SOUdIOFOI ey A ININe,| IS SOAP PT B 9quie aJIYS uN Jed JYNDQSUOD oIPIO Ud sag}os9unu 

Jo yedroutid 93x90} np sdosedas a9 JUSAIOP sayURLOdUII 3a sonSuo] suOTyenbe sa] ‘ajduts soedso un e sipuodsaii09 
JUSAIOP 9]X9} d] SUBP SaIg}ORIeD Soy] sno} stew ‘TediouLd 9}x9} 9] SUBP JOINGI~ JUSAIOP SoyINOD sa[NULIO, so] 
‘aja (.)80] 19 (.)dxo SUID spaUUOTOUO} 

SoJOQuIAS So] Nod Jes ‘ourejUOD UONeOIPUI ouUN,p sULOW B onbITeI US soUTLIdUMTI JUOJOS SanbyyeUISYyJeU SoTOQUIAS So] 
‘S9[SIS Sa] 19 SUOTIRIAQIGE Sa] ‘soded Sop seq Ne SajOU Sa] JOA 


uoHoepey 


“QUINSaI 9] SUBp sonbreWaYyJeUL suOISsseidxo 
So] JOA “S9[d S}OW XIS B SION op IAIns oydessered un,p gsoduioos suinsar un Jed JODUSWIWIOD POP 9}Xo} 97] 


guinszy 


‘aTyderisorqig eB] Japsooid STU SJUDWOIDIOUII Sd] DIAINS POP oXouUR 9INO J, 

"O}X9} Np Uy BY B anrered JUDATOP SJUSUISIDIOWIAI SOT 

"OyXo} 

np osed osorwioid ev] op seq ne 9}0U 9UN SUep JOINSIZ JUSAIOP INajne onbeyo op assoipe,| 39 (SuO] ne 3109) WOU aT 
‘sotudoidde say) sap jue LIOd Sag}O1SUINU SUOIIDAS Ud SOSIAIP 3.99 JUDATOP $d}X9} SIT 

‘Moyne jno} sonod % | SUIOW Ne, p SOSICUL Sop SOAR Jd JNOYed SUSIPAJUI 9]QnOPp & SILI99 9.19 JUSALOP Sd}X9} SIT 


uonejuISIIg 


(Os 


ce 
Ie 


‘sonbrydesd 19 soynuo} Soy Inod astnbas a.n9 yretinod Jaided no Jpd uoIssaA OUP] ‘PAOA\ JUSWo[qQeIgyod 
‘Q]X9} IP JUSIOJI.] SP SIOIYOI SP SULIOJ SNOS STUNOS 9.199 JUDATOP SafIVIe Sa] “SNOsssp-19 sjUIOd se] JaJOU sp 39 (Z JN ‘ZTE “JOA 
np ansed e) ajanbua,p sanbiuyoa], ap yus991 OJOUINU UN JOUTWIeXd,p ssatid ‘ayjouinos a] nod a)x9} SOA JaslyeUL] op JURAY 


SALXUL Sd NOILVINASAYd VT INVNYAONOD SHALLOW 


€79 letter mis in wiv eielese(aievniminterere Siote(erarete nieye sia] eiais eerodloiviele’sisra eiaisaisieiersiiecne Aguatoyya Joyeo13 Joj ssan3 P Zuisn -UPOUI jewuou oy) IO} s]eAJoquI 20uapyuos aseyuuys 
ANITIM Ulqo"y 


E19 TE Lik Bete m ener reece een ences ewer eens ene renee s ase ee een eae e esas ene esse esses ese ees eeE ses ee eH ee ess eeseeeeeeeseeerrereessssessseesenseessseees s1oud SuIYyo}eU Ayiqeqoid uO 
CIqa ‘W A0uen 29 NOIVLS euey-euy 


66S ee ee ee ee ee eee eee eee ee ee ee eee eee ee ee eee eee er rer Sd}eLIeAOD poinjoniys Ajpeosudijauou UM UOISSdIB9I ayyuenb poiysia 
IAM SULA 29 VAFMAVE-NVIAITVS see 


LES Beet eee eee newer e nese eee ee eens esse ee eee eee ese esses tes ees esses tseeesneseseeeenseenseesesesees suljdures Jos poyuel ul IOUS SuIyUeI JO g0uasold ay} ul g0udJaJU] 
AANLZO WO 
19S eee eee eee eee eee eee eee errr reer Creer eee CeCe Ce eee eee eee cece eee eT eee ee eee eee eee ]opoul Ayisuajul jeuonipuos dAISSOIBIIOING 34} Jo SulyINY9 d1soUuseIp uO 


IT 3unoy Ie 27 NOMA Vous UR Teg 


Ivs eee rere rere ere ree eee ee eee eee eee eee eee eee eee eee eee ee ee eee ee eee Tee ee eee ee ee eee eee Tee eee Ty sosuodsal podrjap Uva susisop ulo9 paseiq aatjdepe A|qnoqg 
NVHO'S XM 8 DNNAHO'H US ‘ONWVHZ UX-!T “NH 8ueyzlo4 


IZS eee ee eee eee eee ee eee eee eee eee ee eee eee eee eee eee ee Cree eee eee ree ee eee eee eee eee ee ee eee eee eee eee eee ee eT UOTJELIA JO S]DAQ] ajdiyjnw yim ejyep Suiddens}oog 
HS1T4M ‘1H UV % ONVd UeUuZ “A TAIA “V JoudoisuyD 


SOS isd 01d. lps we 0b \P h 0.0.0 0/5 Gl bia 0/A 6 o'0,p 0'b\b.9 o.0.6,06)6 b.016'8 #16 00 1b10,6.0:0.0 6. 28.016 010.0 0.0'4 0 A606 610.0 Wale o0.c.0'¥.e 0.0.6 alae. /e'e6.0 0 vipinv.c0.s!0'b,0:6,c act emo weet esasive [apow AOYILI uoppry Re ul $9}e1S OM} 1oJ Bul}sa |, 
NNVWZ1OH OffH 22 NNVWANNVG Wor 


€8bp Sarees tala cle aisie aipis'= hia icinla's'plaieialo pisivcs'ginistelsisibid eleivieigivis elaisisia's sisiaiarstetsta suOolINqIysIp 10318} jeloues piemo} Zuipeoy “SUIYLIOS]e syodonayw Jo Zuljeos jewndg 
TWHLINASOU S £04326 29 CAVAAA e4UaIKW 


8007 24quiad9p/1aquiasaq ‘p “ON ‘Og AUINIOA 


CO pene ais pike Ber BIEP PIIOSUdD YIM UITJaFI] pojsn{pe-Ayljenb uvaus Jo JOPOU UOISsaIBII1 9Y} JO} DOUdIIJUT POOYTayl] [eduidwgq 
ONVM UnNysUOH  OVHZ uenyor, 


Cpr wie'a'e,o 6\n\siuin\e/ele;a 6 a b[s:ain\e'd aloin'e'a[s.o(aia sin /ala|eia'nin'a «'s\e\e/s imam ina\e e\nlela-s|b/ale\s ©,c[ala'a nia eja(e's)e[u[ala's'nih a/a/ein(s’a\e'u:a n\p[s(0;0 nivle ¥ieiu\s sb v/elo leis o'n'ele'e n\uleis « 6i0)a'e eleielsle-elvisisre bie S}ysIOM pooylyayl] aatdepe dLjoweIeduoN 
ALNV Id slosuesj-ueos 
L@p BORIC OOCIS OOO OO DIC O OOOO CO CIAO OOO OCOD COCOONS UICOOSCOOOC ICO O COC ETI COCOO DOO CONIEO TOC. OG UOCOCOOO COICO DOO UOU GUCCI SOOTIOOOUCCOUCUC  COUOOOOC COOECO STU] uo1j99}0p UMOUY UM uoijduinsse 


e[ndoo ;eULIOU a}eLIRAIQ dU} JOpUN S]UdIDYJoo0o UONL]IU09 dLaWeIeduOU JO} UOONpal seg 
ATDISATHSALSOUON “UY AYTPA 2 NHO OvNeH “AIN 197 


I Ip Benen ee eee eee me eee eter eee teen eee ee eee eee eee eee eee ee eee seen esses eseseeesesesesessseene SUOTIBOIJISSE[D Areuig 1oj JUSWIDIIITe JO SOINSvOUI paseq-uoyejndod uO 
SCaYVMA UOC 7 NOSTAN ‘d 29> 


TB A TD ea SEES a atin LA SRSA TS « TREE Oe EPCs suSIsop jewndo-q jeruanbas poqoinsa. Ajyeursieyy 
SNAIM'd Se[sn0g 7 NILAWA-NILUVW [nee “OO TVAIs-ZAdOT snnsof 


oe EE Bee eae acd ee teen UOINQLISIP oN[eA-oUIII]Xd dIVURAIG B JO UOHOUNY aouspuadap dy} JO} JOBUINNSS ueisokeg Vv 
NOUAAd slosues{ 2 FLLOTIUND uous 


69€ Cem ere eran seer eee e nese seen eens ee een eee eee ees seen eee ee eens eee eee eeee eens ee ese eensenseeeeeeeeeeseeseeseeeseee suolouny aduapusdap spueyoid jo S$10}BUWII}S9 uonsafo1g 
SUADAS URYOL FY NOTTIND APUWV “CUVLATTIA-S Td Hpwy 


GCE eee eee eee eee eee CeCe CeCe Cee ee cere eee eee eee eee eee ee eee ree ee ree suleyo AOYILIY dUIT}-SNONUT}UOD ul SOUT} pasdeya jo sisAjeue uvisakeg 
CGaYVHONS 'V WRN 8 Vale dass “AV Ce 


Ire once eee e emcee tees mute een eeeeeeasseensteeeaneneneeseeeneeeseceseseeerscceneeseneeeeneese seer eee ceeseeeeeeeeseecneeneseseuseoeeesons SouOUsIA [e1aAas ‘pep jeo1Zojorq Surjopoyy 
CTd14 ‘V J9ydoysuyD 


8007 9Aquisa}das/saquiajydas ‘¢ ‘oN ‘Og DUINTOA 
SHAAILVW SHC ATEVL SINALNOO 


anbiysyeI¢ ap suusIpeuL sNAady BT SIISHRIS JO [VUANOL ULIPEULD 94] 


IS QIS@)SOT 0} PojdaJIp 9q pynoys suondiosqns pue suolssuqns ynoge sasmnbut [Ty 


GO| cere ereeeeeeesssecesseesesssececssnnessscecccsssnnesssecececesnsssceeceeconunssaceceecsunssseceesconnnnssaseeeeeeennunnnasssceeeeecannnnssssseseeeannnnasasces sjeumor JONG Uy 


IS] Saaeeacciciesenlcsecceneseciecseiceeoesccecescieiesieosisececsesiesioccsctcccesessesiscsacdecacescsscousns STII, uopI0yH TWeH uyor ‘yurysng ‘Gd UII], ‘IOYSW vues 
SMOITADY WEMIJOS pue yoog 


6£1 Pee ee meee e nese esses sess esses eee eeeeEeee essa see ss EEE H Hse Se Eee eS eH SESE OSE EEEEH SSE EH ESOS OSES HEE ESEHESOSSSOSOEEEEES SESE ESE OEES SEO EES uewypIog Ty ueTV *juay] orues 
Xopuy sanepiodns ev aeurxoiddy 0} saxapuy IU g pouad aseg JO d8vIZAV payysioAy & BUIS() 


STZI eee eee e een e eee c cece seen eee e ee ececce neces cee seeseensessessecccescsscsscesessceseese INIDY ‘d sUIOID¢ ‘qrues “ai yslysVv uy suoporlK ‘ey “J ur[V 
syONpoOlg xLAeYY dINdI¢g BUIS_] BEC pouonIyeE, AT[eoIA A JO sIsAyeUY BSUIAIDSIIg-ADPALG 


66 See e meee cere cece ence ere e arse sses sees sees sees seees see eees sees esse esses sees ssssseesssesssscssassssessssssssses noy oACUle | eyuens ‘ddrus MOU a) 
spoulo| 1ydeisowg pure syIys uoneindog 


LE see e cece cece rece cece ccc e ee seceececccescesesescccssseceesssssecscees eljseneg aa Joey ‘ul[seOH <2 praeq ‘joyueLy ay uel ‘yyeUUS oP I 
ddIAJOS suOYdaja | ul suondnusjuy pue saioosg Ajisuado1g uo paseg syuausn{py jo uosuedwiog y 
:SADAING BUTTLIC-i3IG-WOpuUeY UI spjoyssnoyY suOYda|a}UON JO BdeIOAOOUON JO} SuNeSusdWIOD 


Cgricccesscseeeeressssseeeeeeeesnnscceeeeceesuenssceeeeessnunneeeeeesneennsnseceecausnensnseeeecaseeeseeeeecageesseeeseeeneeeegsseneseessssseneetee SLNOYA SEE] 
SADAING PJOyssnoH Joueg Ul Wd[GoIg JOAOJ] [PUOIZIY-I9jU] 9y} UO 


Ggreeeseeeeeeestssssnseeeecntssnnceeeeeecessnnnnneeceeesnnnneeeccessaunnnnneeeeessunnnnseeescgaunnenseeeeeqssenseeeaqueeneeesasaesseeeeganenseeeesseensneese ovys unr 
uonenduy JOqusION JSoIVdNY JO} UONRUINSY doURLIe A DLjOWeIeduON 


ic POeeeeee rere eee eee ee ee eee eee eee eee eee eee eee errr errr rere ree ere rere rrerrrr reer reer rere ee eee eee ee eAles RIOSIN ‘Isoyeld evoUOy] 
soo]Jq vary Wopurey pojyeouo0| Jo soussalg sy} Ul UONeUINSY vary [[eUIS 


LZ errr ree eee reer ere eee eer rere ee er eee ee eee eee eee eee eee ere eee rere errr reer eee eee ee eee mq ff youapoy ‘ospupuy yy BN09q9y 
uonenduly y99q JOH Ul syystay J]dures Jo asy—) OU, 


[ooceeeeeeeeeneseceeeeetntsesceeeeeecnsssssnseeeecesnsssseeeeesssssseesseceecasssseeeessnassseneecsessnssensessuessasesseesonseeaeenseesssesnent NOTA “W [2eyOIW 
S[Opop] UOIssaI3oyY IOUT] pozi[eiaUaH Ul SUT] IYSIO A 1OJ SpoIs|] SUISLIOAY [OPO] 


6007 ‘I ‘ON ‘SZ auINjOA 
$}U9}U07 


C7 ceeeeetcrrreteeeeesssctenectnusnaceceetunsseeessnscescegsnnsseeeecsseusnnseenencgenunssacesqannnnanecggunosnstenganuaneerengaanananeeetsessaeet s1oyeIOgRTJOD [eUONp| 


a I 9 occ c ccc nccccccccccc ccc ccccccccccceccccscceesssecesceecescece cess eseeceseseeee sees sesseesssesssesssssssseesesssssessssseeeseeeeeesesesesscceesseesseses AYSAOYNII}D | XoTV 
SUOISIADY dy], pue SULT JUSUNSIAU] UT AjITBUOSeaS 


16S occ ccccccccccccccccccccc cece ccc ccc ccccsccn ee esceseessscecssesccesessscccsscsccsccccccccsccsccsccevcscooes Ja3uIS Joues|q ‘roweuyyeq soe ‘seg Aouen 
gsuodsauon Adams OIparg 0} SuII9U0Z) daj}sIOO] BUIS/) :Po}S919}UT JON Ise JO ‘Asng Oo] ‘swa0u0D AdeAU 


LLS Terre rr rere reer ee ee eee eee eee eee eee eee eee rere ee ee eee rer rere rr rrr rere rer eee ee eee eee eee ueulyIq VV uog “UILIe|AI qoqezi|q 
{SIOMIG IBeIDAOD Snsuay sonpoy pue AJNUSpP] YdOYD IBVIAOD [PUL & S90G 


ie S S etc e ccc cc ccecccecccccccc cet e ese e sec e cece rene seneccee esses esses eceeeeeeseee eee s eee ee sees essen eseeseseeseesee eee seeeessssssssesssessssssssssscseseeseseesesseeceee sueyzZ uny-!J 
Suljduies osNeurajsAs JO soondelg UOWUOD dWIOS UG 


Ibs eee ee eee eee eee eee eee eee ee eee eee eee eee eee eee reer eee eee eee eee eee eee eee eee eee eee ee eee eee ee eee errr reer eee eee eee eee eee eer ee rere err eer rrr Josiog 6) SOAK 
JUDIOYJIOD IULD dU} JO} UOWeUINSY doURLR A UOTeZLIeOUTT pure ojluyyoer JO souaTeAINbY oNo}duIAsy oy) UO 3JION VY 


Pera ae RigteiGe Aesth aise OSes tS ds han len, Pe ee eT rere ae 
SUTUMUL |, IYSIO AA JOJ Spo SuIseI9A VW [OPO 


CIS wis eisie,ejeisie c(elejolelsielela s\elecie sialnieinis dciciocieis'eieia signs one's c'ae(eeiasisinies,aie neve waisteleiets oleae aicldncises vaiaisie sie sieisieise's'ccienisisce nates cenincdcic cance escice nage caicencecacrdseset ccs se Jopulg Vv plaeq 
UOISSNOSIG] 
LOS tee e cece cece cece eseeeeseese eer er er ece esses eseseesee esses eseeseessessenseseeeessesessessesessseesesescsseses ueyyeunysey “A OT[IAU |, ‘royuoyos jJorueuEN 
UOISSNOSIC 
Cé6r Coen reer ccer ener cece ee acces eee ae eeee ees esees sees ee ses ee esses ress ee esses ese sees sees eeeeeesseeee eee eeeesseeeeeeseeeeeeseseaeeeseeseeseeesesssessssssesenesesesees ysueIpas ydasor 


sannuend uoneindog aul. Inoge sdUDIIJUT JO} spoyydJJ URISaAeg JO on]eA dy) BUISsassy 


8007 “PON “pz QUINOA 
$}U9}U0Z 


‘pieog [eUOp_ oy} JO SIOqUIDUT puke S2dIOJO1 JUapusdapuT Aq poMatAal ATSNOIOSU are 
syduosnuew |[y ‘poziseydurs ore suonezruedio [eonsHeis JayJO puke sdoyjJo [euUOTeU Je SONSNes Jo UoNONpold sup 0} yUOUTIOd 
Sanssi Joyjo pue Adojopoyjow AdAIng ‘suoneordde pue ASojopoyjouw [eoNsNeIs ul sazipelsads yey) AJIapenb Aprejoyos ke st SOL 


UIPIMS SISHRIS Aq poysTqng MIAIY [euOyeUAIJUT UY 


SOLLSITLVIS TVIOMAO AO TVNANOL 


+ me 


ality 
wos ses’ = | " 
; a sie on ae 
B ath # nna 
Aythle 
® Bes > 


Es pink iOpen ol yu Bina ge nowane wednle smunpe tbe 
! Sib OLB E PET Ce Gi, eis 
em de BRYA TON fea 0 Gone byte 
} fen Mid ay puting jer 4 ‘pp eines fy eu 
| ae MSHA ID OH. Pune OK 4 


binges 
Ty th 5 Ra TAB utile qo 


| Goa bina q. ype papict APfIDKET 5 calibers ys 
peur ie cae ger mess smsapG edampre BAcE mi S 
Tl Sof Sear 


tay Fits faiyf6 kets qaue cocins, ens: pO 

| nts, sot Bed we a a tee 
a ace ee LENO bes Matic - 
uy US te 1 pia eh Fats 6 eet arn 

PeMncibsoe? pan tite Gh mine Attra plaka 


| Mee I PS PROSTE WRice yh SUITES Ghar Ge PSB 
i es ee iu? Fees ight, HouF*Sdnw- 


wee - 


a i. piel cxbpiem\yee ee Leese pyre ROG 
' so, vs 
; pau gs zoures agiatinie Cl iGe AM raat eriptpiat 


abate" 1, SRS thnbies oir oath sialraprepectes 
RSE Fy aexage? ay! Abtapparen tq ucgn TRS SESTIYB ET 
Oe whys) wee) FS ULST PAO SA GS sors) (sky "aie Re yy 
Se gyane sayiginaqecgeliqg byxtriajey 26 miei VEL O- SR meer 


y i Me ae WT pripitit D Saqvines Tq ws hreR A) 
ie ir em eae Li? Aik = “tite chee ght bith ld 
ie 4 tou sega Ma ANY Yuk oe a Lay ie a a oe 
er alos § e Say ip wae oho! | at, Sy sesh ys oa a | 
Mod Es sats ee ani prom = zu te Sapte ct ne 
eres OG, TG) CT) rato, > 
" o Miron ypnyy ry an 
i eee 
w= 3 a. a 
sean) waive Liab teh 


Liigp gph kl 
was 


i st a “i 


Se 


547500,’ q a4 ; 
4 i - 
pasa 
iM ~ i. 


f 4 


oT et 


i =r 
He 4 /~ 


ce eli eng 
i , Pe WaeA Ue aprtg | ions Bi 
it - ied ae Seam 
Spi.) se 
7 fy tare BWI ops, NucsopNR ® 
Sony PUPAL oH its Naber My 


Hy Wares: Re na 
fey! quky ane 


Tt er fh, At Pe iki) neh be 


ABBA ct a \ we ae were 


13% Gir e i. Hi) fu | 7% Say & j 
pres, rae \ hei fdas Sibine ehiet wey 
BoA TCR I A Pree sess use 


ae hei 


Deane 


By ae we “ open d peigl) sre! 
erate lg bi iWon se ca iS 


| 5 hos nk Y or a 


viven at YY Rae Whe’ 3 
PaaS 4A8e vost aavestepe aay gprs, 


Line ee ae ne. - ‘ca Lisl 14 ey Tine 


Poa 


me oi ugh apt a) 


“ryt aCeat | 
| 


aaphyret PASH 
Cyan Gf a 


me yap ein eipe 4 reciente ja) aye i Ghee Sit s-139 
see ee Qaonkaare’ ass gAipos a3 Fs ee a 


pm Bhepyeilehgie O°) Wy Ow 32. 


*y 
( We 5 be V4 1 Fr 
BRR 1eihMLe CAN RITES Ba Ht van Thee Case) (2 
BAe to NTHAL| ry es cits Nth T@forvis CY 7A) LO aan hy 
. 4) 4 vit sh ay b o3:'¢ et re Vetros 7 
ThONy Pe TG eee pee yer» ea 
VeVi eerie OT pel RR ils Akh at ae | 5 
ay bol: demir i SN A, Hy}, aD. -" ery 
ie fie? Gol 246, BASH age HOREN ayy 4 it 


ih iy I it oN SANE AAO. yy Crea FS gta 
LARA SPR DiS PORE Ae YAR AMT GMIOR, PT ET ‘ 
each By HAMMAN eel A G81 pega Ais fit 
Gahae 92 Se LIOOG (1G Rn ah IPC TOG 6 
PARLE 0 i ae at Ti La GORI MEN hy Aah RRC 
ASL FONG reROpIEg pA ROSS «ic OE REM MGT Fy 
SGI Dy MP RTRG HM HLLIARS (GH, HK COBH FP 
ARBOR bi Fh biDKip aa eal CItEN BAGH a de. 4 FUIWEG 
ay SRW HeE “binaguge tuba, Aiba IGN US idcsch 1S 


- 


i> 


anZojejeo ne X-[00-ZI N ‘epeueD onbnsneis 


‘g0uelLy “AASN] ‘anbruyos} poddey 
‘Anajpsyyn | ap uoypjuaunsog - aaqyuinba aspuuopjiyunyra p 
FIND SVS o4opu vT (yOOT) ‘A ‘NeipleL yo “GS ‘neassnoy 


‘Ylpseg ‘sousypjg Aaaang ul AqjonG uo auadsafuoy 
uvadoing ‘oseyoed Suijdues, Yy oy (9007) “A ‘SILL 19 “VW ‘TR 


"16S-696 ‘871 
‘aouadafuy pup BulUuuD] J }0I1SIDIS fo joudnor “Surjdures pooueleq 


sopun uonewtxoidde soueuea (S007) “A SILL © “O-f OUlAeG 


T16-€68 ‘16 ‘Bylwawuorg ‘poyyout aqno 
OY Surdwues poourjeq wary (pO0Z) “A “PILL © “O-'f ‘aTTHAeq 


“19-€S ‘17 ‘sousyplg jouoypjindwoy “‘Burjdures 
peourjeq Jo wumuosye yey W (9007) “A ‘SILL 2 “D ‘teaneyD 


‘sued ‘onbysneyis aisojopoujay op soounor “AASNI ‘Burdures 
poourjeq Jo} soieWl SYS MON (S007) “A “SILL 19 “DO “eaneyD 


ELI-6€1 ‘IZ ‘2oues, ‘sue ‘SapOIPIA| FASNI PUES ‘tojduo 9 
anjeu suoljnueyog (000Z) “I ‘SWI 39 ‘W ‘oURSUYD “5 ‘a[/epinog 


a1ydeasolqrd 


“SJOMUSUOD SUOTISABINS Jd SdIe}UDUTUIOD SIMZ] Inod 
Q190SSe INIJIPd UN jd SINIJVUTUILXD Sd] SIOIOWIDI INdINe,T 


SJUIWIITIAIVIIY 


‘SINSLIDYN xXNvaed ap jalqgo J 
BIO} ISeSSLUDR .p oseyd ve] e ddIDOSse SOURLIPA ET ap aiduio9 
Ud osud eT “SOURLIRA dIBIA BR] DWIIYSa-snos sayooidde spnuL0j 
RB] 19 ‘JOA op oseyd x e JUOWOANRIOI ‘o}URLIOdUIT snyd sIOje 
}So oSessLua}ye.p oseyd ve] e ddI] VoURLIPA eT ‘saaneordxa 
JUSWIDSLOJ JUOS s8eIqiInbs,p sajqeuea sa ‘““d s[qeueA 
B] Op sed os] sued ‘JOA op oseyd ke] ke SdID0Sse SOURLIPA 
RP] JURADP JIQIe} jSo adessLUOye,.p oseyd ke] ke s9190SSe 
SOURLIVA kT IS ajdexd UOTISIOOId e] ap sydoid jsd (SQ0Z) 
AML 39 ataoq sed sgsodoid soyooidde sjnuuo0jy eq] 


Zeb 


e] 67 LC 6S aayooidde “18 A, 
a! Oc 9¢ 09 SUOT]e[NUNS “Ie A, 
(01x) (01x) (01x) (01%) 
tf yeloy = yeroy = *A ejoy = [ejoy 
OS =U SZ =U 


33¥SS1119}}8 Pp Saseyd sap UNWIWOD Ud asTU 
IIAB JIQIINbs IyyeAjs adepuos ap ueyd un snod xnv}0} xnap ap 
UONEUITSI,[ Ip SBd aj SUBP 9aYydOIdde JdUBIALA Vp a[NUA1OJ eT] 39 
SUOTJR[NUNS YOO QO Jed sQuuop sdueLIvA eT] 24jUa UOSIVARdWIOD 


Z neayqe L 


‘(Q[) ua souUOp saydoidde soueLIeA 
OP INULIOJ BT DAR SUONLINUIIS YOO O] Jed s9UUOpP soULLIeA 
RB] aeduios 7 neajqe} oJ ‘adessLuaye.p soseyd sop uNUIUIOS 
Ud dSTUI ddAe dIqI{INbs sIPWeys o8eM) np sed a sueq 
“ayes onbeyo suep dxf o]]Ie} ouNp aie} 
-uauig{ddns a,u1eu0d e] anole uo Jed ayewdo-snos jsa onu 
-9}01 UOTNIOS eT ‘asodoid suUIOZ]Te | SAY ‘oseIqiInbs.p 
iNejop I JR WWNI] ud sseuUOTTHUeYDI,| JOUTULID} B DYoIOYO 
uo ‘jolIp siqiinbs ose) np seo oJ suep : aseSsLUIDye.p 
odej}a,] op juotaoid uorstooid ap oyod alosay eT] “syuayeA 
-inbo juOos yop siqiinbs ose} uN ddAR Jd DUIILIOS]e 
dNOU SdAB snuUd}qO sje}[NsdI soy ‘soaeol[dxa udiq UOS 
aseiqyinbs .p sajqeuea sa] IS ‘ayWeNs sdepuos op uryd oy 
aonb sjeynsoi sowigur say auUOp ssodoid suTWIOS]e | ‘ayes 
Jed oyuejsuod jyuoWaAneUTxoidde jsd jOIN}UI,p sd[qeUReA 
RB] IS ‘sarsayeys xnop sap ainayjiour ey soAe oqevieduros 
ysd gsodoid asepuos op urd a ‘seo onbeyo sueq 


pS‘l 867 €9°€ 00°9 € ued 
Or'l ZO'L SO'€ ler] Z url d 
8r'€ S67 che 509 | ued 
( 01x) (01x) ( 01x) (,01x)  apou9W 
tf \¢ tf \¢ 
[B}0} “ABA = [BOP UVA =—s- BJO “AVA = [BJOP “AVA 
0S =u St=u 


IS¥SSI119}}8,.p Saseyd sap UNWWOD Ud JST 

J9aw QIqtInbs syyeays jo ‘aaquinbgs ‘ayyesjs adepuos ap sued 
sa] Inod sajqeiiea Z ap [8}0} Np UONeUII)SI,| & 9gIDOSSE JDULLILA 
[ nes[qel 


600z uin[ ‘ajanbus,p sanbiuyoo | 


"SnU9}qO s}eI[NSdI So] aredwIOd | Nes]{qei a] 
“SUOTIE[NUITS YOO OI SP dSeq eT ns sayooIdde jsa ¢ 39 7 sueyd 
XNB I9IDOSSE SOULUPA PT “JUDSUIIJDOIIP dg[NoyTeO Isa | ued ne 
I9IDOSse SOURLIVA BT “a9UIBDU0D UONEINdod ke] op amojyegye 
LQ uN. p dapedaid so [OA ap aseyd onbeyo ‘soiqiinbs sasem 
So] sueq ‘a}eys onbeys suep (7 yUOWdANdedsal) [ aI]Ie 
op UonRoo]T]e oun ouop v uO ‘aIFHeNS a3eI Np sed oJ sueq 


‘asessiuoye ,p 

soseyd sop unuwmu0d ud asIw saAe ates 
“x yo ‘x “uw saqeuea saj Ins giqiinbs ase : ¢ uel 
“x ya 'x ‘uw sarqeleA Sa] Ins siqtinbs aseim : 7 uel 

‘ajes onbeys suep 
g[]duns oilojegte asepuos saae dIjHeYNs o8eIl} : | uel 


: asepuos op 
surjd ston uoyjas sayese saqyiqeqoid e sayrun (Q¢ = u udu 
-saqoodsal) $7 = U ap UOT[HUeYDS UN sUUOTII9S UD SA 
yo '€ sojqeueA sap [e10} Np UOT}eUIT}Sa,| B dSSaINIUI.S UC 

090 8 
jeda yuowoAneuxoidde Yy UONeUTULIA}9p ap JUAIOIJo0o 
un Jouuop mod aisioyo ,d soueteA oun jo ‘¢= “A = “ 
‘00S = “0 daar “(pT[) Jed guUOp sa SA ap smayeA soy IOI 
-9U93 Inod gsT[yn ajgpoul a] _“,d soueLIvA ap 39 () sUaAOU 
op s[eULIOU UOTNqU)sIp oun uUOCTeS saiguas yUOS LU soqT 


(1) Wt At x 2g 4 “m= 


Q[2pour J] UOTaS 
ae19uas yso “4 aqeuea eT ‘ayens onbeyo op ulas ne 29‘ 
B [ed JUsWOANeUNTXoIdde ~y UONeUTULIA}9pP ap JUAIOIJJo09 
4 Ee 
un Jguuop Imod atstoys 70 sourLeA oun ja 4 QZ = "0 
done ‘(¢[) sed guuop 3sao '{ ap smaea say Jolgues 
4 
Iod gsI[yn s[9poul a] ~{O souLLIvA op jo Q sUUAXOUT 
op s[eULIOU UONNqNSIp aun uOTas saouas yUOS 43 say 


(€1) “49 re Alyy _ \¢ 


g]epoul a] uoTas “7 ayeNXs 
B] Op ulas ne agi9uas 3sa '{ ayqeLIeA eT] ‘CZ 19 p Sanoureied 
ap euIUIeD UONNQLsSIP oun uoTas sosi9uas yuos ‘x yo |x 
So[qeuRA sa] ‘ploge.p nop “x yo 'x somelixne sajqeuea 
Z10 “A yo '€ yQigjUI.p SaTqeURA Z : SayqeLiea fp JUPUI}UOD 
}0 ‘oT]Ie} DWIQUI ap sayeNs ¢Z US soUUONIUed “QNO | JINR} 
op a1ulyy uoNeindod oun pioge.p jno} suoIOUSS SNON ‘odeU 
-UO][WULYIE Pp SWIYILIOZTe aNou ap sooueULOjJod so] 19}89) 
Inod suonejnuns red aspma syNod uN suUOSI[eaI SNON 


sonbiiguinu s}eynsay“¢ 
(1) aynuLIOJ eT sed sguuOp jo aIFWeYs aseuUOT|HULYS UN,p 
S¥O 3] SURP oNUD}GO dOULLIRA eT ap ayoold vas OUTRUIOp 99 
ms 4 g[qeueA ef op [7 [eIol np 77 uosdwoY]-ZIAIOH{ op 


IMDJCUINSI,| SP DOULLICA BT ‘BIQIe} BIS Z So[QeLIVA so] Ins 


ayes euquinbe eHeuuoj|jUeYyoI : jeAneyo 


andojeywo ne X-[(00-ZI1 .N ‘epeued anbysneis 


[eqo[s sseiqiinbs | ap so1jauaq 9] ‘sayeys op aU] sIquIOU 
un jueyoduios YG duIeWOp uN IMs steur aiayUa UOneTndod 
eR] Ins sed spod ou sdusIIFUI,| NO Jat;noted seo a] sueq 


H I 
10 x 7 ‘ ee Ya Connie Sei ta) = Vp OAR 


%, > 


(Callpage CCN sy, Dor = Cas qer 


Jed 
aauuop 3sa10 67°] 1 red MA yuedejdural ua (Q]) afnuUoy 
R] Op ynposp os sayooidde soueLeA eg ‘sieIq sues 9}Sal 
“7 eo, np *'7 uosduroy]-zAI0H] ap Mayeumsa,] “soup 
-uodapul | sumiddns mb 99 ‘uonejndod ev] ap ajquiasua,] 
Ins yeqo[s aseiqyinbo un Jamsse Be uodse} ap s9UUOpIO0D 
yQJo ua yuos 4%“ “AQ sayens say suep sosem soy 
‘sasodoid sayneys saiqyinba aseuuo]]yueYyse .p spomjoul 
BR] 9dAe onplod jsa sayeNs sayUdIOJJIp sa] suep sasemn 
sop souepuadaput | onb 1930u ap yuepodun 4so ]I ‘aaysoidde 
aoueUeA op (Q[) gnu} eT suep yesedde jI.nb isury 


= Ah 


v 


Ing ly yy S21 ly lay « 


cial aes Seat eee 
CSc anew ey, 
doe ‘9/6 ased ‘(GQOZ) STILL 39 a[IAoq sed ssodoud 


yy S24 


(11) Ce k = me Rasp 


OURLILA Op INDJeUWII}Sd | JostyyN ynod uO 
l i n?1 ! n>?! 
a YU 
: i ! hig | 
ial sat x ares | < - 
A 'e iv 'e 


H I 
Dale Zien ee la ile deus [ease QOAe 


4) Nn? 


(on) (ed — 4K = CDRA 


® UQ ‘ayes onbeyo suep uoj[HUeYsE.p 
Oxf oy]le} oun jyuRIMsse joa sayeys xne sdueUoyedde.p 
SSOLNBOIPUI Sap 32 UOISN[OUT.p say[iqeqoid sop ympold 
3g] Jed soguuop sajqeuea soy ued anne.p yo ‘"Z salqeuea 
sa] ued oun,.p 101 JUOs aseiqiinbs,p sajqeueA say] ‘ose 
ne yuowsqejeaid s9ussou0d uoNR[ndod ev] jWoUIoMO\egTL 
yURL Ud sdJUOWISNe JUOWAOJ aNQ jnod sIdoNnus,TJ “syOJ 
atdoqud oun daAk d9ST[VAI JS | DWIYIWOS]L,| op [OA op oseyd 
anbeyo 1s “(SQOZ) aTILL 3° a[tAoq sed sgsodoid soueuea 
ap s[NULIOJ e] op opie.] e sayooidde ong ynod souvLRA eT 
“(pOOT SITLL 39 ayaa /9) ederquinbs _p nejgp 
9] JaWUN] op Uye oMeoUT] oUTWeIdOId UN .p UdKOUT Ne gsTTedI 
ang nad ja “4A aeys onbeyd op MOLIDIUI.] & OxIJ dI[Ie} 
ap ayUTeQUOD ve] JUR}DOdsSaI Ud SSRUUO[[HULYDE,| JOUTULID} op 


921 


angojejeo ne X-[00-Z1 oN ‘epeued onbnsneis 


youulod ¢ adeia,q °z adeya,] ap onsst.] B 900dsa JUDUIO}ORXO 
so 7X safqeueA sa] ms oseiqyinbs,, onb omsse mb 930 


n> Ya, I= 24 Yap APY 


(6) MCS pre ne oe ree 


SUOISSAIdxX9 SdIOIUIOP XNap $90 JURIedUIOD Ud 39 


Ya APY Vy, A? 


(9) uonenbe | ap apre.| 8 WeNNgo uo ‘7 adei9,] ep onsst.| V 


tog ha eney n> yy APY 


ae i deka “ints eacat < 


suoIssoidxa $990 JURUTWIOS Ud jo 


Ya) I= tu/4n>¥ 49>y¥ Yay 1> tu>0/403¥ 


mnaege qa (eG alan Sg 


anb onbrydum (9) uonenbe | ‘adeyo ..,[ 91199 op onsst,| 
V ‘[ odeia,] op yeynsor ne poddel red jueuuonpuosep 
ud ay0adsal jsd UOISNIOUT .p sayt{Iqeqoid sap Uw mada a] onb 
ainsse ‘| odej9,| 9p anssi.] & NUD}QO }e}[NSdI Ne JUSWdTTOUUON 
-Ipuod % UOISN[OUT.P saM[Iqeqoid ap m9}99A 9] JOST 
"Sa][2 aUA Pp sUNeYO sUep dXIf d][Ie} 9p UONIPUOD 
eR] Wuajureu op jueyouued jo ‘ayens onbeyo e sg100sse 
ayurequos oun 7x oseiqiinbs.p so[qeueA xne yueynole us 
d9ST[VII JSO [OA ap oseyd apuodeas x] ‘soumWeIdoi1d xnap sop 
unoeyd sued (9007) FILL 3° Ie op oseyoeg Sutjdues 
UY ey suep e[qruodsip jusursyes9 iso [I 39 “(SOOT) IIL 
}9 JOANLYD MOA ‘SYS O1OBUI 9UN,p SULIOJ snos suTUUeIsSOId 
a2 eB OWIYWOS|e, 7 ‘suonejndod sopueis son ap Ins soiq 
-I]Inbg suo][HUeYdS Sap JUSWId}DAIIP IoUUOT}OaI9S op jowUed 
39 ‘uoneindod ey] ap ayyre) ey ap aonb snyd puadap ou jnojeo 
ap sduia} 9] yUOp siqyinbs aseuuo][HUeYoS _p opide1 spoyjoul 
aun juasodoid (9907) STILL 30 eANeYyD onb suojoN ‘uONr] 
-ndod ey op aye} ey op anes ne jouuomodoid yuowaaneu 
-1xo1dde jajja ud jsd O1OeUI 94390 Op uUOT]NOEXxEa,p sduwid} 
YJ ‘S9SU] | IP JOWIDUT JIS 9] IMs s[qruodsip qq) o1MeUI 
R] JURSI[N Ud BdepuUOS ap soseq sopuLIs ap suep soiqiinbe 
suo][QURYIE,p UOTOIas eB] Mod (pOOZ) NeIpse yy, 19 Nevassnoy 
Jed sgsodoid yuowayesd jsd aleyIuNs spoyjou cup 


Fal = GA 


‘uOIsaI anbeyo suep uo][WUeYOS ,_p 
Ox} opie} oun pied onne.p jo ‘uorsai-1adns anbeyo ins 
solelixne sojqeuea anenb soy mod jeqoys aseiqijinbs un 
ued oun_p “JaInsse eB JIIIULUI ap SAUUOPIOOD JUOS sa8vIT] $3] 
Jo ‘suolsel-lodns g ua saadnoisa1 939 OUOP JUO SUOISaI Sa] 
‘UOIS91 onbeyo suep sa[qeueA anenb sad uojas aseiqiinbs 
D9AB ‘UOISOI B] UOTIS dIWeNS ose un smeounied mod 
g1qiey don jsa saguuoj[nueyse saytun,p npusye siquiou 
9] ((88e,p soyouey sion uUo]as Joao 3a ‘ayqesodun 
JOU NUdADI) sollelixne sojqeueaA oayenb ap _ osodsip 
UO “PUUOTDIIIS SO % Q UOIIAUD,P UOT]HURYOS UN ‘syURIIQeYy 


Sél 


000 OOI ep suloul ap saytuN sap uoNe[ndod-snos ev] sueq 
‘sayeIM soyUN,p No soulegin soyIuN.p ules ne sodnoise1 
SUISIIO,| & JUOS S}USWIASO] SOT (ODOT) SWTIAA 19 SUTSUYD 
‘g]Jepinog suep saylep\-Uo]]NUeYY,| 9p sdepuos op ued 
np og[jlejap uonduosep oun eIOANO] UQ ‘seseUdsUI So] Ins 
sajgnbua saj nod as8epuos op oseq op JURAIOS 19 “666 IP 
JUDUIASUdIOY I] SUBP QUUOT}DEI9S ‘s}UDWI9SO] Op UOT]HURYDE 
un jso oeyAy-uoTHueyoy."T “666] eeul-uoy[Hueyoy | 
op somewud soyun sop uoloajas ey Inod (gasuy) 
sonbiuouceg sopmq sop 39 onbnsnesg e] op jeuoneN 
WHYS | .L aasiqnn opoupur oun,p omdsur,s sumyji03ye J 


‘Tu uoIsnfour,p 

sayiqeqoid sap dace ‘444 uoNe[ndod-snos anbeys 
suep ox} a][Ie} op UoT[HUReYdS UN IBUUONIEIaS ‘¢ odeiq 
“ adeig,| 

ap oNssI.] & SaURISOI SoyIUN sop {4 s[QuIOsSUd,| IMs 

“Yu uoIsnpour.p saiiqeqoid say 39 7x oSeiqyimbe.p 
so[qeueA soft oak “JOA ap aseyd oun dJasteay 7 odeig 

“47 ayeys onbeyd suep yusUWIepusdepul 

“Ty uoIsnjour,p sayiqeqoid say 39 *x a8eiqiinbe,p 
so[quuea sa] oaae ‘JOA ap aseyd oun Josteay “| oder 
adessLuaye.p soseyd sap unuIWOS ua 
astul oak ayes guquinbs sseuuoyyueyoY : 7 auypuosjp 


“4 90 }Sd [OA ap aseyd 
J]J2ANOU 9}99 Op onsst.] eB SagjafaI no saguUOTTHURYDS 339 
a10oua sed yuo,u nb “77 ayeNs e] op soytuN sap s[quIEsUa,T 
45% Tu) =, 910U JSo SoyURISAI SIUM S990 9p s[qQUIASUD,| 
Ins JOA op aseyd aiatmiap oun soide sonusjgo sayiyiq 
-eqoid sop mayooa oT ((*u/tu 1z ue bat Sars 137 ‘1) 
= "x jo ‘sagjofor no saguuo][nueYyse 919 o109Ua sed jUO,uU 
nb sgyrun sa] ousisap 4 no ‘4°"("u) = wu Joa op saseyd 
S90 Op INSST,]| B SNUD}qO sayIqeqoid op sIM9}d9A So] H* T 
=Y UA GnI aie, jou uo :sayens FY sap sunoeryo 
Ins yUsUTUUepUddapuUT dasI[BaI 1Sd [OA ap aseyd ou) ‘oseuUOT 
-[WUeYOS Pp SANLUIAITe apoyo ouN WUMOJ | JUITVUIOS]e,] 
‘yens onbeyo suep ox aijie} op ojjao aonb sajurequos 
sanne p Josodu.p syowued aseiqiinbs., aonb mod sqrey 
doy jso ens red uonpedal ek] no seo of sue “aXxI} d{[Ie} Op 
aseuUo][HUeYSE UN JoINsse.p uge ‘oseiqiinba.p sayutenUOS 
sop omed yey "u ayqeuea ey aonb asoddns ug ‘nuuoo jso 
Souelfixne sojqeuea op ((7z ‘"u) = Xx majooa un ayjonbey 
suep yo ‘49 “ “'Q soyens Y ua sednoosap fQ uonry 
-ndod oun.p sed a] suep neaanou ke suodeyd snou snon, 
“QOUBLIPA 
op SIvIq Sus Md}eUISa UN IUd}go MoOAnod ap uye sunoeyo 
Inod Z ai[le} ap uoljyueyos un.p oyun] eT JUOWTeIOUds 
JUeXT] 9S Ud a]eQS onbeyo suep soUUOT}O9I9S UOT]NUeYIE,p 
o[[!e@} Be] oTIMpel eB yUSTADI INb 90 ‘aouauTyIEd es IoIOI[OWIe .p 
uye uonendod ej op uy say asednosap un Jasiyyn,p 
yuonboyy jayjo ua 4sa [] ‘ag}00dse1 ang ynod ou ajuTeQUOD 
91190 ‘sed suIeyI9d sueq ‘ajeNs onbeyo suep uolNueys9.p 


6002 un! ‘a}anbue,p senbiuyoo | 


uontmedss ey e wodder sed ajqiey ane yop aseiqiinbs.p 
Sa[qeueA op siquiou ay onb asyuou! (/) UoNenbs,| “uauTas 
-1oaid snjg ‘UoT[MULYIE,p a]Ie1 ey @ yoddes sed snptatput,p 
sIQUIOU s]qQIe} UN Ins dI0d asessLUaye.p aseyd e] Is aJeNs 
anbeyp ap ules ne 9}0adsal ualgq vias adeiqiinba | ‘| uoNoas 
ud agquasaid sipneys siqyinbs aseuuo][HueYyos .p oinpaooid 
PR] UOTAS Q SURp dUUOTOITAs sd § UOT[HURYD9,] IS 


ISESSI119}}8 Pp Sasvyd sap UNUWIWIOD Ua 
ISTUL IIAB IYHL.YS aiqiyinbs sseuuoyHueyIy = °*Z 


‘syueyodu 
}So sonore ysa A nb uoTNHueyse.p ayjiey ey onb 9\dadsal 
xnoIul jueyne.p vias mnb ‘ayes onbeyo suep jneunxoidde 
adseiqiinbs un omouwued op asejueae,| JUsWIaTesa ayuUasoid 
SWIYJUOF]L 19D “JOUT.P I[Qeuea ke] & SddTdLI0D UdIq yUOS 
aseiqiinbs .p sajqeuiva so] uoNetndod x] ap ayquuidsud | Ins Is 
a3e.ys onbeyd suep oxIj a]]1e} op ae] SOAP DIJHENS aSepuos 
un e yoddes sed adroroure yso uorstooid ey] ‘ayens onbeyo 
suep JusuTUepusdapul sauUONdaTas snyd s1Oye JUOS dU suo] 
-ueyoe say ‘ajyens onbeyd ap ules ne oayleynos uontpedal 
PR] op yous yoodsol ay jURyoUIad ja sSaIsIoYyd sayqeLeA 
sop mod uonendod ev] ap sjquiasua.] ims aseiqiyinbs un 
yuvimsse ‘oqny np spoyjeul ey op aidepe aseuuoy]NUeYI9 .p 
SWIY}UOS[e UN ajURAINS UONDES eB] SUep SUOSOdold sNnON 
‘ayes onbeys suep 
UOT[HURYIE.P IXIf a][Ie} op ayUTeUOD eT] onb ayne uOTIpUOD 
oun Jasoduit .p ajqissod sed sioye js9,uU [J ‘ayes anbeyo suep 
Z Ble} op uUoTTMULYSS uN IJoUUONDIIIS & d}SISUOD a}UBINOD 
onbryeid sun ‘ouly soy uodey ap satZHeNS sd UONLndod 
eR] IS : oseiqyinbs., onjouued mod ajqrey don 3sa ayes 
Jed uoniedai ey ‘seo sutepyiao suep siepy “UOT[HURYDE,p oI[Te} 
PR] JULASP JIQIeJ 9jSo1 9svIqIINbs.p sa[qeueA ap s1quIOU J] IS 
ayes onbeyo ap ules ne 93d0dsa1 ualq elas aseiqiinbs, J 
“X o3vlq 
-lyinba p sayqeuea soy] ed sonbiydxa ualq yso A jQ19}UI,p 
g[qeuea ke] a}eyNs onbeyd suep Is s[qley eios uosdwoy] 
-ZWAIO}{ 2p IMayeumsay op soueuea eT (*u — J)*u 
= 7g sastyn e aystsuoo ajduns snjd ey “'g syuaroyyo09 
sa] Inod suoneurxoidde smarsnyd yuasodoid (¢Q0Z) SITLL 3° 
ayaa, “u/!d tuj'x Ig "971 (tuyix 'uy'x '97?'Z) = "J no 


ty, “N>Y 1=4 


@ Crd - OF KX = Cea 


juango uo “syHeNs gaquinbs 
aseuUO][NULYIE,| Op Sed a] Sue] ‘“sIql{INbe osepuoOs op urjd 
un Inod uosdwoy]-Z}AIOY op IMayewjsa,, Mod soureueA 
ap seyooidde sjnunioy oun juastnpsp ud s{I ‘uatuosstod 
uejd un.p seo oa] suep ueAnnu uosdwoy]-ZiAIOY 
op IMoayewnjsa, op onbynojduiAse syyeuuou op ssoujodAy 
aun snog ‘uossiog op urd un.p jouUONIpuod 2] SUTWIOS 


ayes aiquinbes eBHeuuol|UeYyoS : yeAneyo 


anZo[eje9 NV X-[00-ZI1 oN ‘epeued anbysneis 


NA ojo ynod uoIsnjoul.p sayyiqeqoid sawigui say yuLyadsa1 
}9 SOTQVULA SOWIQUI So] Ins soiqiInbs asepuos ap suryd so] 
Twued syeurxeu aidonus e siqiyinbs ued ay aonb yuanuour 
(S007) PIL 39 eI[Aed ‘0 = (0)307Q UoNUaAuOD eT daAe 


N>s 


((s)d) B80] (s)d “C- = (d)] 


Jed s9uuop 3sd_Q uoTeIndod e] ms 1uTap 

(.)d asdepuos ap uryd un e agto0sse aidonua,p ainsoul e'7] 
"$a9100dSa1 JUSUID}OBX9 JUDIOS sayuUN sao ap 4°"( Tu) 
SO]JOUUONIPUOD UOU UOISNIUT.p sayyIqeqoid say aonb syzlos 
ap *4°"(%u) uorsnjout.p saytfiqeqoid sa] d0Ae ‘[OA ap aseyd 
PR] Op jeyNso1 ne JUSWITJOUUONIPUOD ‘saguUOT[HURYSE yWOS 
sayuRj}sal saytun sa] ‘aseiqiinba.p sayqeuea sap uosduioy yp, 
-ZNWAIOH, Op AMaeUINSd,| 9p JOA ap aseyd ey] ke a][OUUON 
-IPUOD JOURLILA P] JULSTUMUTU Ud ISeUUOT[HULY | JOUTULID} 
op jouulad |I no suas ne yewmndo jsa ued a_ “sayuejsar sayrun 
sop , uonyendod ey ms ruyep jeumdo asepuos op ueyd un 
‘JOA op aseyd ek] op je}[NSaI Nv JUIWIdT[aUUONIpUOd ‘ITUTJap kB 
SIO[V 9JSISUOD a8eSsLUae p oseyd v7] “sa9}dadsa1 JUSWUA}9exO 
ano snjd yusanod ou aseiqiyinba.p sayutequos say puenb 
a]9Ie S JOA op oseyd ey ‘aseiqiimbs.p sajqeuea op siquiou 
3] ousissp b no ‘4 ayens anbeyo suep snpraipur b 
snjd ne mod Jayouey e 338d [I.nb amnsse (/) uonenbs,q “JOA 
ap aseyd er] ap onsst,] @ saayoadsaz Jusuayiejied yuos a3vIQql] 
-Inba,p Saj}UIeNUOS Sa] 39 UOISNTOUT,p saytyIqeqoid say aonb 
yuamnsse (9) 19 (¢) suonenbs saq ‘JOA op oseyd ev] soide y 
gyun | Inod asud a109ua sed js9,U UOISIOap PI IS JUSUIDIOLNS 
[ 319 Q agua astidui0d 3Sa 39 Sg}dfaI 4S9 dI[9 IS CQ ‘SaUUOT}DaTIS 
soy oun] Is | NVA *yu : [OA ap aseyd eI Op jeyNsSaI 9] 
QUUOP | 2 IN9}99A JT ‘JOA ap aseyd eI & g190sse aseUUOTTH 
-ueyod,p ouistuessul a] snos souesodsa.] susIsop J no 


(L) Dae teen ad DIS) 


(9) PO ee) 


(S) YW = CWQhZ 


: SOWUBAINS 
suOHIpuoo sg] jueyuaA [T ‘O] 5 Hig (tees yf mmajooa un 
47 ayes anbeyo suep yuanqo Uo ‘JOA ap aseyd eI Op onsst.] 
V ‘uoneindod ey ap saytun sap uN, | JUDWISATIUTJEp I9"WeRd9 ._P 
NO I9UUOTIIIS ap JUoUaMOWwaTe aplosp uo ‘JOA op oseyd 
eR] ap adeja onbeyo Yv ‘asesstuiaye.p aseyd jo [OA ap oseyd 
sogjodde saseyd xnap ua asodwiosap as al[q “Isloyo yUoUIE|q 
-pjeoid = uOIsNjoUT.p saytfiqeqoid op nof un yUoUIa}ORxO 
yuejoodsa1 ua ‘sayqeueA op onbuosjenb siqwiou un ims 
soiqiinbs yuswsAneutxoidde suojyueyos sap IsuUONde[9s 
ap youtiod (007 SILL 1° AT[!A9q) 9qND np spoyppu eT 


(p) phe ot 


vel 


‘Ip Tesus@)aaneypo : jawunod 


‘gourly ‘zug QL] S¢ ‘UUeT Joy ap snduieg ‘Jeoseg osIe[g 2M ‘JVSNA/LSAUO ‘aenbugq,.p onbysneig sp ssloyeloqey ‘joAneyD suineyjinH *| 


anb {2} alIp-k-}$9 9 ‘aiqiInba yuowWsAeuTxosdde 
UOTTHULYIE UN IaUUOTDEII9S ap JUdWIaTeIOUaS OUOP jso JNOOf 
-qo,J ‘a]qQIssoduit sa nb 90 ‘,] = xX dy[epow ev] JURJUOSoId 
SMpIAIpul ("hp 19 Q= X guyepour e] yuejUesgid snplarput 
€°S JUeUD}UOD UOT]WURYSS UN IOUUOTOOTaS op Werasoddns 
‘x g[qeLeA ke] Ins gigqitinbe ‘sajes9 soyyiqeqoid e “OT oIIIe1 
ap uo]NUeYSE UN IJOUUOTOITAg “Q aI]epour ve] JUa}UAsoId 
uonendod e] ap snprarpul ¢¢ onb a]a} ‘T 19 CQ ‘Souyepour 
xnop & X I[QeLIeA oUN dIUTJap so sjanbsa] Ins SNPIATPUT OT 
suuanu0d “7 Uonendod ey aonb ajdwoaxea sed suosoddns 
‘QANON 39 JUdWIZTeINUNS ynod ou sIqiINba yUoWIA}9exO 
uo]]WUeYSE UN ‘sayuTeQUOS ap onbuosjanb siquiou un.p sed 
J] sueq ‘siqiinbe aseuuoj[Hueysep Jolnorped sed un duop 
}So ayes onbeyo suep oxlf aie} op aynens ose oJ 


“> "53 

(¢) BERT MEE TE 
B JuasInpal as (Z) suonenbs say ‘adeiqijinbs,p aqeurA 
g[NIs kB] Sd UOISNOUT,p ay]IGeqoid vy] IS aIIp-e-}s9,9 “Uw = x 
no Joynomed seo a sueq ‘oseiqyinbs,p sajqeuea sop 
]210) np UoHeWMNsd,] nod anu ouop jsa uosdwoY ]-ZAIOH 
Op INI eUINSd,| Ip DoUBLIvA eT ‘Ssdg}00dSaI JUDUID}OVX9 JUOS 


(Z) ses Tea 

suonenbs sa] Is X so[qeuea 
Sa] Ins giqyinbe 1p ysa_(‘)d asepuos ap uryd aq ‘saiqyinbe 
Suo]]HUeYyS9 Sep JouUONde]9s ep joutiod Inb (pOOT IIL 
}9 STIABC) eqnD Np oumpuosye,| op opie] e g1OTpUIe ang 
nod ayeys onbeyo op ules ne oseuuoT]HUeYo9,| ‘uoNe;ndod 
RB] Op NprlArpul onbeyo nod aes yueAe s]qruodsip jsa sarely 
-ixne sgqeuea Bb ap (x “ “x) = x mayo un IS 


‘ayes onbeyo ap ules ne juR}suOd JUoWIOATeUITXOIdde 
ysa "u/"A Is JuawIa}Oexe snyjd Jo19}UI,p d]QeUeRA ev] Op piesdal 
N’ S9UISOUIOY JUOS Sa}eNS Sd] IS J]QIe} VIIS DOULLIBA 390 


iba moral 
(1) {z = a\™ a 
: ApUNID-saye A-UIS 


OP SOULLIVA Op J[NULIOJ e] Op aple.] e JUANGO,s UosdwWIOY] 
-ZWAIOF{ Op IMayewumsa,] ap soueuea x] ‘ameleos jsa 4 
= ‘Z no Jarnonred seo a] sueq °Z (a[]aL0}99A) s]qeuReA 
eT ap “7 IMs [ejo) gf ousisap *Z7Z= HF no “7 IK 
= Li Sieiqusues! oumnse) js !ujaze SK sig noite ane 
Tu"Z SYR = “7 uosdwoy]-ZtA1opy] op Moyeunss,T 

‘Saroja yuos “u ayes red UOT]WURYDI Pp SaI]1e} Say SoyNo} 
anb gore] ap ayins ey suep suosasoddns sno “7 a7ens 
ve] suep uonedal ey ousisap “uv no YS T=y ‘tu 
=" "0?'Z sanommed ua sioje & UC ‘axXf a|[Ie} ap so 
()‘d ued aj ‘*74 aeNs anbeyo ap utes ne.nb asoddns ug 
"2 1(%y) = yt yo °"("u) = w eI9}0U UC ‘UOTTHURYD9,| SuURP 
JUIUIAUTO[UOD SsagUUONdITaS JUSIOS 7 19 Y Sa}OUTSIpP soyuN 
xnap onb /u aytypiqeqoad ey] sa ajutof uoIsNoUT,p aytIqeqoid 
PR] 19 ‘UOT[MURYDS,| SUeP saUUONIETIS Hos y 9sUN,[ onb 
"u gutqeqord ey isa y 9yuN,] ap UOrsNpoUT,P sitTIqeqord 
ey] Hy = y “d syuepuadsput asepuos ap sured 
Sop UOTAS saUUONOaT9S JUSWIBATIDOdSaI yUOS HY“ ‘T = Y 
“e suoywueyos sop sayjenbsay suep ‘sayeys sogjadde 
H“" ‘t= 4 ‘‘q > suonejndod-snos jy ua sauuorned 
189 Q uoNnrndod x] ‘ayNeNs ose) uN.p sed a] sueq 


4N>1#¥ 1=4 


fp") x ites ee (*) 18 


uoHINpo.yUy “| 


"sopesoul 


souiqeqoid e adel) { uoNRoyNeNs foqng np spoyjou { ayewxew aidoyua {Jye}01 sBeUUO][WUeYDA : S99 S]O| 


‘sasodoid yuowiaye39 Js9 JIeIOI adeUUO][HULYD9 UN,p seo ne uOeT|dde sug ‘uoNe{ndod x] ap a]quiasua | Ins sde1qiinbg,p 
So[qeuea so] Jed sgnbrjdxo udiq jso 1QI9jUI_P d[QeLIeA k] IS adIOI[9UWe Jsd UOSdWIOY]-ZUAIOH ep InaeuNsd,| op uorsiogid e] 
‘aye-Ns onbeyo suep Z a][Ie} op adeIt} UN, p JUeWOdUIT Jal;noIed seo a] suep anb suoNUOW SNON ‘a}e-Ns anbeyo op ulas ne sdxVy 
uoneooT]e oun yue}dedsaI ud ‘uoNe[ndod ze] op s]quiasua,| Ins sIqlInbs uo][NUeYSa UN JaUUOTIII9S ap JURWOULINd a3eIN op 
spoyjsur oun 191 suosodoid snoy ‘oaneuxoidde say uodey ap aonb gy0a0dsal vias ou adeIqiinbg ,| ‘sayes souleyi9d suep oI qQIey 
1S9 UOT[HULYDE,_p UOIUedaL EY] Is ‘UepUuddaD 1OIDIUI,P I[QULILA Fk] & S99{9LIOI UIQ JUOS SdIIeI[IXNe Sa]qeLIvA Say Is slogid snjd 
vlos uosdWOY]-ZiAIOH Ip Indyeullj}sa | 19 ayes onbeyo op ules ne siqijinbs ase sed sioljawe ang jnad aseuuoj]NUeYog | 
‘nplaiput onbeyo inod sajqruodsip jUOS sodIel]Ixne S3|QeURA SOP IS "}QI9IUI,P 3[qeURA P| Op plesoi ne sauadowoy uaiq yuos 
SdJVS SO] IS JOoMp odes) un e yoddel sed ojInpoi sioye jsd UOSdWIOY]-ZAJOH op INojeulysa,| op soueLIeA eT ‘suoyeindod 
-SnOs sap INS dyes ssepuos op ued un Muysp ke ajsIsuod d}uULINOD onbyeid uN ‘UOT[{WURYDS UN,p UOT}OI]IS B] Bp SIOT 


puINS ry 


, BaneyD ouneyiny 


IHVYS g1qiinbg ssvuuolypyueysy 


ec 


enBojeyeo ne x-,00-ZL .N ‘epeued anbysneys 
LZL-€Z1 “dd ‘| WN ‘SE “IOA 
600Z uInl ‘ayenbua,p sanbiuyoa 4 


- - ieee 


TAU 
’.' 7 oH aoe oe aaa " 
y iitei) au see» BE rar Pe 
, - a) raf) ; ae , \ Mi 
ve eas ” rs yt hy ee. fi 


ss Sar sae 
hel, Mgapgyea.) 
phe seems Worwaetteielbi 


i 
Meh! 


ers 


| SRIANE EE hdareaigt 
vray “Songs yoda stile NRT) SOR 
BR IGS) a ROY 82 DORR 
is bal 
arrest wera Sr Ae ssdiemadie ne 
una. eaeet) A SHPTIC, eRS 


noi 


ve 
so an, 

‘ya. 

“Ura uth 


| > . 
Y Naa “SELES (et: TAN: Caray 
ae a en nog) if 


Mira lis ics | 


ety an a 
: GES TL ATCA, Pater Ory) JO perenne 
Doig om Hunpy Ag seuy deevclbr rien sem 


ements ‘tear? yy Mpeg Shiv way 


' arise yl ¥ paeitig) haidyy Myel Aaity 
bd Th Aethaloeslew Bamypoe po wiowsihae'y 


5 faesan ‘pig Hears “5 eget PS “lecie es eeg 
sy sda pants wep eas 
no 4 


Soh) fo soil Js 


id es nee Alene desi ees uy we fu }) ak 


“age 
“ 


ds Al-gy 4% i. pote “WY ite PR * ay 
‘Kou. WO B00) eR oepodio pur pees 


i TET) ord oy road fi (aie on 


“Sy Ca? erent ‘Tie ‘wreartiieass 
thames an) fp penny Zaamy Bawa” 
0 in hia Ot ee es ee 


Var) 95g POT 
ies es - ‘cat aay oe ——s tsa? 7 “SH 


Feo act) 
anaeee wegen: romed ie o wale 
a eo? S MCR] 3 ¥ a ; “yD mite 


te 
Ei ists 


on 
‘ i on why 


nn : fe ieee Vey ae | 


sn LALLA PRON Ns TES 


‘ 7 te i) 7 =. 
jay ‘inebia » et kiey, 
dive srrspamn oy habdy|} LS del pn * “." ven 


~. 
mf yy > % ag 
. str oe oi aw 
a sctieh eased a "aKa ote ee % shes » 7 
fal tux Mi a Sees es ‘nde fe ee 


Borgia, ' oH, 4) wh shagtizeggs ap ma" J on. 


led es oe Pe Teapr) ae went, wry “eage 


wa 


eee oot": stem ier’ rt fe Aint (2) 2 MPR 
Tinie Anba aps Pe Apoti-oorie yw ann 
rT nie “eal WE eh ry hry ef erm 


eid 
SAPO 
(Wars eye . 


} aoe “y 


Rept be “? y aint orn re (0) pe “MOLANS Ri Le 2 
eats wy “sebvaltyfcea? bh eA My ities! ‘yf “eed 
ts 
We LCN Se AAR een nS Pusey  SPUAGNS wirtafon 
sh aya ee el ee a eee 
meaty eT due um hsp 

Sim pany eB a ee ee ie ey 

' ; 

rh Re > LPTs Rey, Ay! Mii avid aa, 

Py pice’ Fi iA ivy ald oy “ pee seg vi i y *; uy 
My ‘e "eos! 
7 
re LT UORE d = tar ae bE Ne | a) 
me. th Seay 2 " re fn nee "wl 
iweeaieg tk 5 wen Ma's a 
. WEEE. LO os 
if 

“et OORWR IL, y HY Cola <cenngse 6 DL AE PY ‘Tar 
SOE TRS OF) DIY ep! Po my Bes vu ics st 
LOTS SAN ap ATS “MERKEL gris | Gas (nay 


“WiearatiGn) Suis) wep Woe), 
hones ‘ayes ap Lace re ted: bow of we we Mr, WG 
1 SPO aR PY Aes. i We “IE 
pecadty op sob aneaiegr he 2ntus 
a OS Ue ial veg acne Nee ay 
yj eNageoerete, |. op prone) a onbethan 
eanebateanibe sonnet ap BPy /Oacees 
~iftirerng enon tl geryaie Kpre seoRRNR2 stax ENNS yo TO ATY! 
eps facie HSLDA ey MORRO, 487 0 RGA 
loaned pa owiehe ins? Ot Op carrey arte ages 


Way wa A waren 
iv “a 14 * 
ReatTved &ci ee 


vermeiid «prot teri ted 


Ni i Ps 


yt Be PNA arabic aan pomseicarA” NL BINA 


aie, "4 aver SU sik red oi ~— 


at stil eet sae, Mi f 


‘ 
ph ase! comet! 


tie rie, 
» 


St 


anZo[e1e9 ne K-[00-Z1 ,N ‘epeued onbnsneis 


‘WIeplo}SUTY “WIepso}suTY Jo A}ISIOATUA) 
JVIOJOOP IP ISI] “YIIVISSI UOI}DIJA UI S}daJJo SN[NUINS pue seIq 
asuodsai ‘seiq asuodsaiuoN ‘poysosajul you we | (pQOZ) “Y WOoA 


VS 
-€7 ‘Q| ‘Sspoyjayp pjaty ‘syuspuodsaiuou BulAsaIng (SQQZ) ‘| ‘doors 


“WAY NeJOp/LOOTA2tYose/S1 
adeduoissnosip/ysievasos/UapoyoWl/NUsU/TN-]U/JU'sSqo MMM //:dy 
ne giqtuodsiq “sinqlooA ‘SqD ‘ZO0OLO UolssnosIp ap 
I[DILY ‘SOPOU! UONDI][09 BIVp PUR SdISd}e1NS YIOMPIIY JUIIIJJIp JO 
UOSLIedUWIOD 94} 10} s1OJeOIPUI-Y “(L00Z) “J ‘UIqQqoD 19 “'g ‘Ua}NOYIS 


LoL 


UY IYNLJOP/POOTZ 
/jotyore /siodeduolssndsip/yoseaso1/USpOuoW/NUSUI/ TN -[U/[U'sqo 
MMM //:duYy gay ous ne s[qruodsip “sinqiooA “SqD ‘100r0 
uoissnosip ap Jardeg “8661 (STOd) SuoNIpUoD SulAry pjoyssnoy 
uO AdAING paye1Baj}U] DY} Ul SseIq 10} JUDUNSNIPY (POOZ) ‘_ “UaINOYIS 


POX 

MIN “asuuds ‘soysyejg ul sauiag siosuudg “‘Burdures Addins 
poysisse-[apopy (€OOTZ) Lf “UeUNDIA, 19 “Gg ‘UOSsUdMS “dD ‘Tepules 

“purlsuq “IasoysiyD ‘suos 29 

AayiAy uyor ‘ASojopoyjay AdAING ul sauag AaiA\ ‘asuodsoIUON 
ym skaaing ut uoneuMsy ($907) “S “Wonspuny 30 “-_ ‘jepugs 


76S 
“18S “€9 ‘Dylajawolg ‘eyep SuIsstu pue sdUAI9JU] (9/61) ‘A’ “UIgny 


600z unl ‘ayanbua.p senbiuyoo, 


"8S7-EPT “OUT “SUOS 7 AdTIAA WYO : 10 
MON “(COPWT VEY 9 sung “Ty “ueWTIG “vd ‘seA0InD WY 
‘Spq) asuodsaauoy Aaaing sueq_ ‘sisAjeue aatsuayaiduios 
V “sj]od 1x9 ul ssuodsaIUON (7007) W ‘UeWwyapy 10 “W' ‘apa 


‘OIP-16€ “€ “EOI ‘Uuayng jor1s0joyodsg 
‘OZIS o[dures JO JOaIJa dU] :sisAyeue 10}9e} AIOJLULTUOD UI Saxapul 


W-JO-SSOUPOOH (8k61) ‘AU “PIeUOGSW 19 “WL “Ve “AH “YsIeY 


"VSN SAN “OU ‘SUOS 2p AZT, 
uyor : YIOX MON ‘soNsHRIg pue AyyIqeqolg Ul SaLIag AdTIAA “eIep 
BUISSIL YIM sishjeue [eONSNeIS (7007) “A “UIgny 19 “WEY ‘op 


‘$97-SP7 LY 
‘anbuysyojg ap ajyouoijousajuy anaay “sine jeoysyeys yuouND 


‘[]] suydures saneussaiday (96/61) “J “aTJASOW 39 “A ‘TeXsMDy 


€TI-III ‘Lp ‘anbusuvis 
ap ajDUuOYDUAAJUT ANAIAY “SOYSHRIS BUIPNOXd sINjeIO}] IYNUSIOS 


‘JJ Suydures sayneyuasoiday (G6L61) “A “TJASOW 19 “AA ‘Teysny 


Toe 
‘Ly ‘anbysyolg ap ajpuoipusajuy anaay “dinyelayl] S1jWUaTOS-UON 
‘] Suydures sayewussoiday (P6161) “A “ASTJAISOP 19 “AA ‘TeYSMUYy 


LISS 
‘spoyjayy Yyoavasay Aaaing “eLIa{LId [eUId\UI YIM seq asuodsaruOU 
yun Jo jUoUWISsasse UY ‘opisul Woy skdAIng (£00Z) ‘N “IaT4OH 


‘O8E-69€ “7 JOT ‘suouny payuy) ays fo jousnor 
Joousyng “uoysanb siseq ay} BSuryse Aq seiq asuodsaruou 9y} 


Sulonpal ue Buuo|dxy (pR6l1) ‘Oe ‘WaysyIeg 19 “QI ‘UasIsy 


‘SP I-STI “p9 ‘ApuardwnG uoiuidg a1qNq ‘Karis 
auoydaja} jeuoneu e ul ssuodsoUOU BuUIONpol Jo saduanbasuoD 


(0007) "S “Hessaid 19 WY ‘seaoin “y INYOY “D “TNA “S ‘e122 


“LUI-61 ‘7 “67 ‘sousuwig payddy ‘eyep edu08a}e9 Jo sannuenb 
odie] SuNeSNsaaut Joy anbruysa} Aroye1ojdxd uy (0861) “AD ‘ssey 


‘OI-€ ‘1 ‘1 ‘spoysayy youvasay 
danny {jo Ked syoyya ino oq :soyei yoRjUOOUOU pur [esTyaI 


AdAins ZUIZIWNUT, ‘(L00Z) “D “IP[aasoo’y 19 “Y ‘SIQy “q ‘YsaMIIN}] 


‘67S-LIS ‘Tp “UOUDIoOssp 
JDINSIDIS uvdiueup ayy fo youdnog ‘skaains ajdures ul 


asuodsaiuou Jo wajqoid ay “(9p61) “H'A\ ‘ZIMINY 19 “HY ‘uasuey 


VS IOEPEq [sueW 
:yIOX MAN ‘suoyeindod syuy woy suyjdwes (1861) +f “Yyeley 


LE-Z ‘89 ‘Apdaqaondy 
uoluldg oygnd ‘suoisiosap uonedionied Adains Ul 4sdoJ9]Ul 
d1do} Jo aJ01 dU] ‘(POOZ) “S ‘OXdIq 1° “S “‘Jassaig “WY ‘Sd9A0IH 


“VSN “AN “PYPUIC “NOR OE Ne BZ 
‘gsuodsauoN Adaing pjoyasnoy uo doysysopy [euONeWIOIU] ,,/] 
ne a}Uasaid jusUINDOG ‘sISATeUe-eJOU Y :seIq BsUOdSdIUOU UO Saj}e1 
asuodsazuou jo yoedun ay (9007) “A ‘eAdyayAeg 19 “PY ‘SAA0IH 


“$L9-949 ‘S ‘OL ‘Aj4aquonG uoiuidc o1qGNg ‘sKaAins pjoyssnoy 
ul seiq asuodsauou pur saje1 asuodsaiuON (9007) WY ‘S9A01D 


‘purysying ‘ufiiog ‘sepia A-1osuuds ‘suonediisse[o-ssoi9 
Jo} UOTBIOOSSe JO SaINsea|] (6261) HM ‘TeASMDy 19 “y'T ‘uewpooH 
“‘Us}J99H “SAO “€SH 


-L00Z-OOS anbruyos} poddey “YyWA op fiq Surmodweem spoul 
-poxtul yusWIIadxy (9007) “AA ‘S[9ZIOA\ 19 “G ‘UassuRL “Ss ‘sfamnoy 


ongoyeyeo ne X-[00-ZI .N ‘epeued onbysneis 


‘OUO/MIPH 2 ueudeyD 
‘deyys}oog ay} 0) UONONpouI uy (C661) “fy ‘TuRYSQI] 19 “g ‘UOyY 


‘8ZP-EI bp “9 “Ap4ajaonG 
uOluIdG I1/GNq ‘WUIWIUIS JaUINSUOD JO XOPUI dy} UO sadUeYS aIeI 


gsuodsal JO sydaJjo oY (0007) “YF “1asUIS 19 “S “IaSsaIg “Y ‘uTIND 


‘BINGIOOA ‘Sq_ ‘UOIssNosIp op JoIdeg ‘s1OyeoIpul 
“Y Jo uonepyea yeourdusa uy (£007) “A “uainoyos 1a “4 “uaqqoD 


‘TesnyWoOg 
‘uogsly “LOOT WOR GZ Ne EZ “LOOT IS] Ne Muasgid yUsUIND0q 
“BUINDS UONII][OI eJep SpOWl-poxtw ek Ul ddUdIAJUI [eONsNeIS 


‘(LOOZ) Usp URA ‘f ‘ayRIG 10 URA > ‘JoyIOg “g ‘UassUeL ‘4 ‘UdqgoD 


‘O9TZ-1S7 “€ ‘p ‘SouSIIDIg joIDYYIOC fo JDUANOL “UONRUINSS UOISS3IB9I 
ysnoi} seiq asuodsazuou jo uononpoy (8861) ‘OL ‘WeauysTyIEg 


‘6S I 
-6b1 ‘pl ‘anbysijojg ap ajpuoyousajuy anaay ‘sasodind yenuaiayut 
Joy o[dures & Jo ssauaanejuasoidal JO ainseaul VY (900Z) ‘°S ‘oulLEg 


“OPT-BET ‘7 ‘LOT “UUayng jooisojoyodsg 
‘S[OPOU JeINjONS Ul soxopul i sAyeIedwioDy (0661) Wd ‘Jopueg 


“VSN ‘AN 
“Quy ‘SUOS 2 ADTIA\ UYOL : YOR MAN ‘SoysyDIG puv yiqoqosg 
ul Saldag Aapiyy ‘siskyeue eyep [eouosaeg (Z700Z) “VW ‘nsasy 


o1ydeasonqig 


“syjoMysuos 
SoeJUSUTIOD ~=39 «sSUOTsassNs sme] op simayeurWexd 
xnop jo g100sse Inajoepal ay onb isute ‘pyaasooy yaeH 
‘uassuer WOlg ‘SoAOIN Gog jWOIdIOUIDI sane say] 


SJUIMIIIIAIWMIIY 


‘yred anne .p 
“y Inayesrpul,| ap sadAj-sinaua say 19 “ued oun.p ‘sourerrxne 
sgjqeueA ap aiquiou a 39 xIOYd a sud UOTRID Pe 
IdIPM9 SUOLIADP SNOU JUSWISUISISIOI] “SoyoI[dxs (WUOWISAy 
-eulxoidde) soulloy sap 3mua}qo suOAap snou ‘sorsayeI)s 
SOSIDAIP Op UOsIeIeduiod ve] suep Jude snd sor un juonof 
Y smeyeorpur say onb suojNoA snou Is “IQ “SsArTeU sUOTeU 
-1xoidde sap aonb yuouuop ou onbiyswrered uou dens}o0q 
g] Jed sonuajqgo sinaua say ‘apore yuasoid 9] suep gsi[yn 
Y Inajeorput.| ap odAj-Inoua,] ap onbuosy} pnoyed a] so 
dYoIayoo op xneaevy sInjny op jofns owerxnep up) ‘OLIeEUZNS 
amd a] snos asuodal-uou ap sieiq ne oyu] oun ouUOp 
Q]]2 ed ‘[eUITXBU NOSge sTeIq ap UONOU ke] I[qQuI9 SUOCAR SNOU 
tonbinod js9,9 ‘snd vou yesay da] OU INaJVOIpUT one UNONe 
‘sqyuenbueul sosuuOp ap sUISTURDDUT 9] INS JUOWOUSIOSUdI 
unone sues ‘uepusdad ‘s}ey[nse1 suoq op sed yUoIOUUOp 
dU Y SINIBOIPUT So] ‘d]QeIOUBI UOU JUIWIIPOJ }S9 19-INJ99 1S 
‘sqjuenbueul saguUOp op UOTeaID op oWisIuRdaUI Np pusdop 
sosIpmea sayqelieA sap asuodal-uou ap sieiq 3] JWastpoid 


sejenbua xne esuodai e| ap a}yIAlje}Uasaudai e] ap sunayeoipu| : WaYa|yjog Jo UBqgOD ‘Uua}NOYyoS OZL 


angojeje9 ne X-[00-ZI1 oN ‘epeueg onbnsneis 


Y smajeorpul soy ayjonbey suep omsouwi ey] ‘asuodas-uou ey] 
dP UODALIOD ap SapOyJoUl Sap sed 3] SUBP dUTWIOD ‘aonb Jom) 
-9afuos ‘juanbasuos Jed ‘suoanod snon ‘sosel]ixne so[qeuuea 
ap gquissua,| op juspuedap y siMoajeorpul soy ‘nusjnos 
velop SUOAR, | SNOU sUTWOD “UepUdday “ITUdAR | SUBP SUOTSSEy 
3] snou onb artessaoou yUoUINTOSqe jso [I ‘UdWIO\sazIUR/Y 
‘ajgnbua,.| ap sulayt sap ajdwioo nud} no} np sed suoae.u 
snou ‘naly Jomuoid uq ‘sojoudoid simoy 39 Y SINa}eoIpUl 
So] JustsUTa[d srpussdurod mod sastessad9U JUOIIS SIPUOJ 
-oidde snjd dnooneaq sonbuidura ja sanbuosy} syosoyool 
op xneaey soq ‘odAj-Inaue uos op jo Y INaeOIpUl UN.p 
onbuidure apne szatureid oun yuanuos ajore yuasoid oF 
‘gsuad SUOIAR | 
au snou onb sapueis snjd juos sadAj-sinalla soy °% 7 
UOMAUS,p InasIey uN e JURpUOdsaLIOD % CG B DOURIJUOD 
ap [PANU] 19 %¢6‘Q OP Bp1O., op yso odAj-mau, TJ 
“SPUUN YOO Of UOMAUSp 1S9 VdH.| 9P UOT[HuRYyso | ep eiffel 
eT % pS op oaneunxoidde masse] oun ke % C6 B JOURIJUOD 
ap dJeAIOUI.| “oyeuIoU Io, ouN.p sdsoyjodAy,] suoWoWD 
snou Is ‘ouog °% ¢*[ UOMAUS.p adA}-IndUIa oUN SUOAIISGO 
snou ‘009 € ep UOTTWURYDE,p o]Ie} oun mod ‘a;dwoxa 
Jeg ‘soaysapoul uojjnueyoe.p sayjte} sap mod sapuess yuour 
-OANLIOI JUDINOWIAp sayja stew ‘oIpusye A_S je]][eJ [I OUTUIOD 
‘qUoWIsne uo][HUeYSI,| Op o]]Ie} ey puenb juonurump sadAy 
-SINdUd sa] onb aUOUW saIs1oyd sapnyd xne uoNneordde J 
“QI QIeJ ISO JOIOUT ,P SUID} Say Inod 9AJOSQO [eUITXPUI NjOsqe 
stelq 3] snyd ‘aAaza 1sa osuOda1 op xnv} J] sn{g ‘esuodal-uou 
ap sieiq un.p onbsu a] jyuasinpor saasja snjd osuodar 
ap xne} so] onb suoje}suod snou uauaTaIMeN ‘adrqiinba 
snjd ssuodai oun e& JUoWoNessao9uU sed jUdUDU OU SdAaTO 
snjd asuodai op xne} say ‘asuodel ek] ap ayAneyuasoidar 
RP] 39 osuoda1 op xne} a aud arejO uONe[aI ouNone 
asIxo,U [Inb g[9aAg1 & Y IMa}eoIpul.| op uoNeordde, 7 
"SoMleSS9d9U JUSD\soyIUeUT JUOS sanbutduia ssguUOp 
op siquioU puis snjd un ‘Uepusdas ‘soyqeyTeA STyNO sap ane 
yusAnod Y simayeorpul say onb suonyjouos snou ‘juanbgsuos 
eq ‘sinajne soo ied saanjoayja sogiteiap snjd sasAyeue 
SOP S}RNSOI So] SdAR OpPIOOUOD Y IMIJeoIpUl.| op IMa[ea 
B] Op UONNUIWIP Re] NO JUdWASSIOINNR,T ‘sInayne sayne,p 
Jed [Iejop Ud soUTUeXd 9}9 JUDIVAR s}e][NSII say JUOP ja 
soguue SolsIWap sao seg-skeg onbysyeisg e& sagsi[ea1 sapmya 
xnop e gsodoid y Imajeorput., anbydde suoae snon 
“Y sunoyeorpul say Inod « saireydusaxa sanbyesd » say 
Ja10qe[9 Inod yedep op yuiod surwios onbindwis uonepryea 
B] oIpusid issne jreumod uQ ‘sajqeiepisuod ane jaaned 
9 sogjomuos sed yuos ou sadA}-sinoud say stew ‘sieIq un 
DINPONUI,P 3IA9 UO “UOSRJ 999 9q “OOURAR_pP UONPOTFNeS 
oun JOXxf slew ‘sjapow! uN JoUUOTaI9S sed ou preLINOd UO 
‘UOT[HULYOS,| Op o][Ie} e] op psesa,] e douRpuodap ze] Joye 
Mod sagsi[yn ang JusAned sajuaptag sayooidde sasioaiq 
‘stvIq UN oIINpogu ynod s[9powl np uoNde]9s ap aisa}eNs 
ey ‘orpuodgi e suorsusdoid sap uoyeumso,] suep yuepodun 


6LL 


JJO1 un juanof samelixne sajqeueA seq Ins saguUOp op 
ai[iquodsrp ey 12 uOTTHURYD9,] ep oT]1e} eB] onb ayyrusts yuIod 
IDIWIP 9d ‘SAYLOIUSIS JSa UOTNQL|UOD eT JUOP SUOTILIOIUT 
S23] aJepoul oj suep juawonbrun juejnole us jo UONBOTJIUSIS 
op neoalu un juesodul ua sonjoajjyo JUoWaT[aMIGey JUOS 
J]Qpoul np jusurajsnle | Ja XIOYS IJ ‘sTeIq op sed yinponut,u 
UOTJOLSAI 990 “yuouONbyjo}duAse steur ‘stda1d suIOUI yUOS 
19-xnao anb anbijdun y simayeorpur say Jowmso inod uojy 
-UPYO9,| B OJIN] aS ap yey a] ‘soyURWOdWT sodusNbesuo0d 
sap e Inb arpuodai e suoisuadoid sop uonest[apour ey ynoyMs 
1S9,0 “uepuadad ‘soURLIRA INI] 39 sTeIq Ind] yUSOUSNUT mb 
sadeja xnap spodui0d UoNeUWIsa INI] 39 SoTOJeITe SoTqeURA 
Sop SOUIQUI-XNd }UOS SINd}VOIpUT soo ‘OUD ‘Sd{[ONprIAIpUl 
aipuodai e suotsuadoid say mod sajapow sap ims oindde_s 
19 UOT[MULYDE UN INS 9pUO}J }SO Y SIMIVOIPUT Sop [NITed IT 
‘prepurys sonbystje}s sjarorsoid 
sop uednyd ey juesiyn ua sainojeo ong juoAned gore | 
suep sojuasoid sonbrydess so] 19 Y IMa}eoIpur, JT “orpuodas 
e suortsuddoid sop simajo0A Sd] SUD DOURISIP Op SUOTIOUOJ 
SoUdIa}JIP yuowsduls juessisioyo ua syimysuod one 
yUdAnod Y SIMIedIPUI sane ,p WoWa][oIMjeN ‘osuodgi-uOU 
R] op uoNIsodwioo ey ap samoajue sasAyeue soy SULITFUOS 
Y Inajeorpur,, onb anuouwl sayjae1 ajonbus,p saouuop 
sop eB uoneoydde,y “moaua sues saguIsa oa yuoAnod 
aipuodai e suotsuadoid say puenb ssijeunou ja 9\oI1d19} UI 
ang jnod ja Japnojeo eB apoB} ysa jI.nb sored ‘moyauwoid 
So gjoe jyuasoid og] suep yop Y IMayeorpul,T 
‘Jatjnored yofns un Ins yueyod saygnbua saydyynur 
ap e no sapoued sajdnjnur op ins soasijeo1 sajyonbua 
sop & ye) JURAR SapN}d,p sIped J] SURP SddNjO9TJO Ssaxa[dur0s 
SasA]CUL P S}EI|NSII Sd_ JOULITFUOD op JUaWoULIOd s]I,s sayyn 
JUOS Y SINILOIPUI Sa] ‘sdQUUOP sap 9]09][09 ap sarlsa}eNs 
no sajgnbua sayuaiayjip sazreduiod mod sajqejeA s[yno 
Sop JUOS Y SIMayeoIpul sao aonb sepr.] 1aAndde e suoysiayo 
snou ‘anbuidura ajduioxa,| ap ushowl ny ‘ayANneiUassido1 
dIjIUsIS «Y» No “Y sInayvorpul saj suojadde snou aonb 
99 op ojduioxa un jsa gsodoid inayedIput,| ‘NA SUOAR | snoU 
SUMO “INdeOIpUl,| ap uOesTyN,| JUowonburdurs 1aysny]I 
J9 SANeUISaIda1 e] ap gjqissod InayeoIpul uN aiMsUOd 
‘gsuoda1 ve] op ayAnejuasoidar eve] op uondaoiod oun 3a 
IsnoNosU JUdWIONbyeUIaYy}eU UOTJUTZSp dun JoUUOp : JOT 
yuasoid a] suep syyoafqo spueis sioy suoAInsimod snon 


uOISsnIsIg. °*9 


“aup yustwuaidoid 
MLNS LP] Op doUeT]IOAINS ap ajgnbua,, e asuode1 ev] ap 
g]J20 onb suu0g suToUW yUaUTaTGISUas sed jsa,U ajOTId Tessa, | 
B aj9]duI09 asuodal ke] ap a]J90 onb sipury ‘asiqiinbs surow 
JS }0]Id Tessa] e oUsI] ud asuodsl ek] ap UOTIsSOdWOD PR] 
onb Issne suoj}e}suod SNON (900Z) T]O9 32 sjaMnoj ap xnao 
sed jUasipaquos ou 9g neajqe} ne sajuasaid sye}[NSoI sa] 


6002 UIN! ‘a}anbus,p senbiuyoo] 


HL WEL (PFE BWTIS “LY SIDE  snjd audi uo — ajoj1g 
Ypsl WEI (SOSI'SL) MLL MTOE SIDE dUSI] Ud — 330] Iq 


%89 %W89 (HTE08) VHS %689 6E1 0E STTBUIPIO 
ac at zy er) y xney ou ssuoday 
OVLVOVdI 


ded tains ja aust] ua sajuesoduiod daav ajorid ajygnbua,, 3a 
yuawanbiun aust, ua sajzuvsodwosd daae ajojid ajgnbua,| ‘asreu 
-Ipi0 9}11NIIS B] AP dduET[IIAINS ap ajgnbua_.| e asuodg.4 e| nod 
d[BWIXeEW JPOAY 39 [PWIXeW SIvIq ‘adJUeTIFUOD ap aT[eAsazUl 
‘Y aAnajestpur ‘asuoda1 ap xne} ‘uoTWURYII,, 9p ael 

9 nesg[qey 


‘oIIeUIpIO a}gnbua | Mod onb sapueis snjd yuos saaumtysa 
sodAj-smoua soy ‘ojopId ressa,] op uoT[WURYOS,| ap ayjiey 
ayed snjd e] ap asneo Y ‘ameurpio ajgnbua,| mod sasarasqo 
smo[eA sop sayooid juos “gq onb sure Yy IMayeorput,] 
‘g[qUIdsUd UOS suUep d}0]Id 1esSa,,| Inod ‘anuosd seg “aileuIpsoO 
a3onbua,] od aonb pues snjd stoy xnop ap snjd ysa_ “g 
[eUNXeUI Nose ste1q a] “Y IMayeorpuT a[qrey a] 10 asuodar 
op xne} g]qley 2] SIOJ e] B gUUOP JURY “%¢ ep syood 
So oJUepuodsauI0a d imgeA eT -OIIeUIPIO ajanbua | mod 
aonb oust] ua asuodai ej mod sqrey snjd sd Y IMajeorput | 
aonb squoUl g neajqr} a] ‘asuodal ap saytjiqeqoid say mod 
ONbNSISO] I[PePOUl J] SUBP SddUUIOTJDETIS 939 UO 39 SaysIsa1 
sop sade p soouedde 339 UO aseusuI ap adAj ap jo u0n 
-esiuegin.p oisop op ‘onbruyjo odnois ap ‘ase .p somerpixne 
So]QeUeA SIT “a[]quUIasUa UOS sup da}0]Id ajonbud,] Jo SUBT] 
Ud d}d9][09 op spow ne saytuN] ajoyId ajonbua,] ‘omeurpso 
ajgnbua,| : sednoié sion mod “y 32 “g ePID ‘y ‘osuodar 
Op xne} 9] “UOT[HURYDE,| Op a][le} e] SUUOP g Nes/qr} 97 

‘yueuu0}9 sed jsa_,u Inb 99 ‘saguuo][HUeYdE SouUOsIad sap 
938] Inod jnoyns jUd} UOTVE}e}SUOD aad “JUSWII[NEs oUBT] 
ua npuodgi yueAe souuosied xne 99}IUN] 1Sd UOsTeredul0sS 
eR] puenb satyejuasaidai sutowl yUdTAap asuodai e] onb saorput 
SOP JUIAYIAI s]I ‘slojaynoy ‘aseusut ap odAj op jo uoNes 
-Tuegin.p aidop op ‘onbruyja adnois op ‘ose,p sorelixne 
sojqeuea sa] Inod asuoda1 ap xne} np sauearun assieq oun 
juaj}e}suOd s]] ‘sonbrydersowlap sadnosB-snos SIdAIp Soy] ZY 
J[QuIS JUSUISATJLISI JSO UONUTUNIpP 399 onb stew ‘ojoyId 
Tessa_J Inod s]qrej snjd so asuods1 ap xney 9] onb yuouUSy 
-NOSs s|J ‘sesuodal sap suoyisoduios sap sauearun asdyeue 
oun e& apgd0id yO (990Z) ST9ZIEA\ 39 UOSsuURe ‘s[TaMNnot 

‘ouleuIpIO ajgnbua,| mod sarasqo tnjao onb ayqrey 
sryd yuowia1939] Jsa asuodal op yeqo[s xne} a] -% CE UOl 
-IAUd,p oiteyuawa{ddns asuodai oun yinpoid e& e199 ‘OV L.1 
@ NO) OVdI.| & Sepeyeor oip yuo SosuUO][HueyoS spyrun 
Sop % OL op seid anb ayrusis ejag oust] ua seuUOT}sonb 
3] [dura Juo sauuosiad ap % OF JUSWII[NIg “o[qQIeJ 1SO USUI 
-onbriun dua] ua ajonbua | e osuodal eT] ‘e[quiosua UOS sUBp 
aj0jId ressa,] @ asuodai ey] 19 JUWIaNbrun susI] US sjgnbus_,| 
mod ajojid tessa, e asuodai ey ‘omeurpio ajonbus,] mod 
asuodal ap xne} sa] DUUOp 9g neaqe} oT jJUOWsQNe OVdI 


angojejeo ne X-[ 00-7] ,N ‘epeueg onbnsneis 


ed yo gtjqnd suoydg|a) ap osJoumu un juarepassod siI,s 
OV] Jed nesanou ap ssysoidde 939 yuo aus ua ajgnbua,| 
® sjuepuodsl-uoU say] ‘ousI] Ua ajgnbua sun.p uaXour ne 
sagyooidde 919 pioge.p yuo Jodtorsed A nod saguuonsayas 
souuosiod so] “o}XIW apoul ek 9}99][09 ap aIZa}ens oun 
SUPP SOPOUI Sop UN,| SWWOS JaWID}UT JasITyN,p ayt{iqrssod 
P| Jon[eag.p Inq nod jreae 9007 ap ajoyid tessa, 7 ‘OVWa] Jed 
sagysoidde juos suoydsa} red sayutofa1 ana sed yuaanad ou 
mb saad ‘OV LI Jed saaysoidde yuos atjqnd ysa auoydsyay 
9p O19UINU 9] JUOP souUOsIod sa] *d}XTUI 9}99T]09 ap apouT 
e ajgnbus oun p y193e Ss |] ‘SUONOUOJ sas y]duraI sorjod e] yUOp 
uOSse] LT & 19 SJLMOVS BL] & SOANLTOI SUOTSaNb sap Ins ayod 
‘snjd jo sue S[ op seg-skeg sap uonejndod e] ap soidne 
d9ST[VaI “OIIVUIPIO 9}LIMO9S ek] Op douR]]{I9AINS ap ajonbua.T 
(L007) [OyeIG 19 Jooq ‘Uassuee ‘Uaqqod JaI;nsuCD 
‘So[[Ie]oPp SJUSUIOUBIaSUdI Sop MOg d}XIWI apoul kB saauUOp 
SOP 9JDB]JOD op saIdayeYs so] JOIpMea.p ona ua (4oj1UOPy 
Mafos) gyIMNd9s ek] Op soueR]]IOAIns ap ajonbua] ve jJyeyaI 
ao]Id tessa un gsijea1 & seg-skeg onbysnes ‘9097 Uq 


9007 9P 9}xTWI 
Ipoul & 9}0[Id Tessa : 9}1INDVS BL ap VDURTIGAING §¢°S 


‘0% ¢ Op UOTIeO 
-IJIUSIS Op [Inos ne JyeoyIUsIs yso [adder ne sasuodai sap 
ynole | & JNNDISUOD Y Md}edIpUT,| Sp JUaWIassto1NNe | ‘ano 
uq ‘aseq op suonsonb sap ayj20 39 adde1 np syooidde | nod 
SUOISNOUOD Sd] JUSULMJUOD Y sIMd}eorput say anb suoje}suos 
snou ‘Vqq.J op tAms np s[dwioxa, op sed a] sueq 


HSE WE (L06-EFR) VSL8 WOES SEL OI 2Seq ap suonsanb + Wqq 
MIS KOS (S68-1'E8) %E98 %S'89 SEI OI Vda 


SEs) Lae nEGRED: on. 


uw 


xney, ou asuoday 


SIMVIPIXNE SI[QVlvA ap Npuajd ajquiasua,| anod 

329 JUIWa[NIS sI[qnd suOYds{I} ep OAIWINU UN jULAY SadeUdU 
sat anod aseq ap suoysanb sat snjd vq, anod 32 Vq4q,| 4nod 
aewixew JIOAY 32 [ewixeul siveiq ‘aduLlUOD ap a{[eAjajUI 
WY danajvorpur ‘asuodai ap xne} ‘uoTHQURYdI,, 9p aley 
g nes[qey 


‘onutumtp “g ‘yJuRoyusis sed ysa_u II 
‘NROANOU Op SIeU “y ap JWOUTASsTOINNe UN sUO}RISUOD sno 
‘gseq op suonsenb sop ayooidde | & 19 VWd4.[ & sUIquIOS 
asuodal ej Mog ‘a]qrey snjd ysa “Gg op Maye eT ‘souRTyUOS 
Op dJeAIOUI.| Op IMdsIK] ey] auSI0UIA} Ud sUTWOD ‘sopueIs 
snjd yuos saguijsa sadAj-smmaua say ‘aed snd yueus} 
-UIPUI S39 UOT[HURYSS,| Op I]Ie] ev] OWUIWOD ‘soseUSU sap 
a]quiasua | nod anb saaja snd dnooneag jso Y MoJVoIpur | 
‘Te1ouas ug ‘atTqnd suoydsya} ap o1dUINU UN yURAR SodeUS 
XN SO}IUW] SO UOSTeIedWIOD 9})99 ‘¢ NeaTqQe} NY ‘JUSUIIIN39] 
onutunp “g ‘oaneoyiusis sed jsou uoNnuIMIp s9D 
‘aseq ap suonsonb sap syooidde,, saae asuoda1 23399 op 
UOSTEUTQUIOD BF] & Wdq,| & esuodas ev] suoreduo0s snou puenb 


Y op uoNNuTUNIp oun suojOU snouU ‘p Neo[qQR} 9] SURG 


sajyenbue xne esuodai eB] ap a}IANe}USSaIdaJ | ap Sunayeoipu| : WaYa|yjJog Je UEqgOD ‘UE}NOYydS QLL 


andojeje9 ne X-[00-ZI1 .N ‘epeued enbnsneis 


WEL WEL (POS9SL) %OBL %9SL PLORI seq ap suonsonb + Vda 


Hob Wr (SLEH7Z8) %1SS %69L PLO8I peddei + vd 
%OS %WOS (L'ZSLL) M108 %7C9O PLO8I Vda 
rahe gaan ite eB yo ney lw asuoday 


SITVTIXNE Sa[qvliva ap Npuajg a[quiasua,| 1nod 
aseq ap suoysanb say snjd vq, 329 ‘jeddea ay snid Wqail ‘Wd 
dnod ajewrxew POWAY 39 [VWIXeU steiq ‘adURTUOD ap aT[vAsI}UT 
Qy anajeorpur ‘asuod94 ap xnv} ‘uoTyURYII,| Ip da19gpuod aye] 

p neque 


‘Of ¢ OP UOTBOTFIUSIS op NedAIU Ne 
g = Pty — Stly 2°77 ayfnu asayjoddAy,| ap yofor ne suut 
jelajejIUN 3sa} un,p uoneoydde., “uepueday juoyoneasys 
as jaddei ne jo Wdq.] & SaeutIquios sasuoda1 say inod jo 
Vda.I & esuodai e] nod feel JOURIJUOD OP Sd][PAIOJUI SIT 
‘ONUIUIp “g [PUITXBUL N[OSge sIeIq oJ] ‘xnop sno} JUd}UNUISNe 
Y moyeorpur,, jo asuodar ap xne} a ouMM0D “Wda.l 
® sjuepuoda1 xne saynofe juos jodder ne syuepuods1 soy 
puenb ajusuisne Y op Inova ey] UoNeIapuod ev] op asneo 
e ‘saJUAINS suONoeS xne sayLIOap sapnje say aueou0s mb 
30 Ud SOpURIS JUSUIOAT]EIJAI yUOS SadAj-sInaLIa say] ‘aseq ap 
suolsanb xne asuodsal ey e sauIquios VW qq,] & asuodal ey] 39 
‘jaddei ne asuodai ey] & soUTqUIOS VW qq,| & asuOodal BF] ‘Vdd 
e asuodgs ej mod “y 1 “g eae ‘y ‘osuodar ap xne} 
J] ‘UOT[HULYDI | ap sosapuod ayjIe} ve] BUUOp p NeI]qQe} IT 


SUPI[CAUL,p SUOT}EdO0][e SAP HOSOI 

aseusul np neAou np sIquiswW UN suIOW NY 

aigunw91 10;duia un apassod 

adeuaul np neAou np dIquiawW uN suIOW NY 

SQ]RINOS SUOT}BIO]]e Sap WOdaI 

aseusu np neAOU np dIqUIDU UN suUIOW NY 

(IM) « nusady Je 1o]durg » a1jU90 Ne WIOSUI }so 

aseusuw np neAoU np d1qUIdw UN sUIOW NY 

IWOUOIN INd][IeAeI] UN Jsd adeUdW Np neAou Np s1quIDUI UN suIOU NY 
Sosse[d [| Ud [e}sod spood np neaaiu ne JUdW9aZO] Np suUaAOW INa][e A, 
ely 

SOSSEID Q Ud adeUDUI ap ddA], 

SdSse]O ¢ Ud UOTeSIURgIN Pp 9139q 

Sosse]d p Ua onbiuyja adnoip 

Sasse[o 9 ud UdAOUI OB 

sopueis snjd sd] So]]IA p Say] 19 QOUIAOIg 

sasse]o p ua sAed np uolsay 

g]qnd suoyda]a} ap o1duNu uN apassod a3eusul aT 


aque, 


(a)}uasaid Is d1teuazied (es) UOS ap ja ASU Np Jays Np gnjzIysu0d 


jsa9 asvugui np neAou aq (L007) ueNoYIS Ja UaqqoD ap 3a 
(L00Z) UIINOYIS ap sapnja sa] suBP SaasTTN sasreyIxne sapqeLieA 
¢ nesjqe 


"SON[IX9 939 JUO SaT]a “UOUTS *% ¢ 
dP UONLOIJIUSIS op [Ines Ne dATeOIJIUSIS UONNGLyUOS oun 
JUaIevUUOPp Sa|[9 puenb ssuoda ap sayypiqeqoid say nod onby 
-SISO] UOISS9IS91 IP S2[9POU Sa] SURP SasNyoUT 919 UO saTTq 
“SOATILIISIUTLPR SOSUUOP 39 SONSIBAI SIOAIP ap jueUdAOId 
uo][NURYoS,| e& Ssosuedde 99 UO sa[qeLRA sag ‘¢ neayqe} 
NB SadISUINUD SoMI[IXNe SI[qvUeA Op I[qQUIASUD,| aST[N UO 
(L00Z) UsInoYsg onb IsuTe “(,99Z) UeINoYog Ja UaqqoD 


LLL 


‘syueyodum uonng 
-LuUOd 9UN gUUOP JUO d1OUNUIAI [RAB UN,p aI][99 39 gI[qnd 
auoydgya} op oouNU UN, p UOTssassod P| Inod |[-(— SMoyeoIpul 
szq_ ‘(Z00Z) uaInoydS 32 uUaqqod sed sasiNN onbnstso] 
UOISSOIBAI Op sajapour so] sueq ‘sanejuesoidol suIoUr 
uontsoduios oun jinpoid aseq op suonsenb xne sosuodai sop 
19 Wda.| ® sasuodal sap uosieulquios ey] ‘sjuepuodsl-uou 
S23] 39 sjuepuods say aud 9\SeNUOD 3] oNjUD00R ‘OUOp 
a « sjlored syey[nsol op siqwiou pueis snjd un» juMoy 
I-9]]2Q ‘aseq op suonsenb sap sysoidde., mod gasesqo 
919 B asIoAUT | ‘atjqnd ouoydeya} op o19urnu op sed jueAe.U 
soseusul Say yUeN[OUT Ud daIIp-e-}sa,9 ‘uoneindod ex op 
a] quiasua,| nod ‘anuos Jeg ‘aseq ap suoysonb xne npuodal 
yuo tnb g1jqnd suoydaj9} ap orouInu Un juURAe s}uepuodal sap 
jnofe | soide saneyuasoidas snd yuataap osuodar ey ‘ jadde1 
np oysoidde , mod aonb syeynsa1 sowigur soy nuaj}qo jUO 
st ‘atqnd orsunu un jure saseusUl xne YUN] UOTTWUeYD | 
mog ‘ajqnd suoyds]a} asp o1sunu ap sed juepassod ou ja 
yuepassod saseusul say nod sourgur say 339 sed yUO,U suONR} 
-E]SUOD S23] ‘IAINS OP SULIOJ 91399 Op seo a] sueq ‘anbysiso] 
UOISSSIBOI OP SdTIPOUI SOUIQUI sap jo SomeI]Ixne so[qeueA 
op gquiesud oUIOWI np jURAIOS 9S Ud (/00Z) UaINoYIS 
39 uaqgod sed osasdyeue 939 ke omeyuoWg;ddns ossuod 
-91 RB] ‘aseq ap suolsonb sap oyooidde | ap seo of sueq 
"SOISIOYD 
SoMelIxne sayqeuea sap yso nb 30 mod sayeyuasgidar snd 
}so joddei np ayooidde | ap 19 Wqq,| op s9UTquIOD asuodal P| 
anb aimyouos e syod jU0 | suOTR}eISUOD Sad “‘sonbNsLg}oeIe9 
sougur xne yey e Inb 90 ua sjuRjsIsiod s}juepuodsl-uou sap 
JUITLIDTJIP SIAUOD syuRepuodsl-uOU saz onb aAnoid sunone 
LOANosep BU [I ‘daNO UA “SdISIOYD soMelIxne soyqeuea 
Sa] ouIZDU09 Inb 90 Ud VWd4q,] B& SJuepuodal sap jyudIayIp 
jedde1 op syooidde | ap sed a] suep smdAuod syuepuodal 
-uUOU sa] onb njouod ke {] ‘anbyysIso] UoIssaisa1 ap sayapour 
OP IAIOS 1S9,S [I ‘osuodai ap odAj 2] aIpoid Mog ‘sonbrurou 
-0990190s 3a sonbrydeisowlap sonbysugjoered ap s[quiasua 
pueis un juesi{yn ua jadder np sysoidde.| ap seo oy suep 
syuejsisiod syuepuodal-uou xne 39 sIaAUod syuepuodel-uoU 
xne Vqq,| & Sjuepuodal say sreduros e& (4007) UaInoYyoS 
‘gjduraxa 
on0U suep soyesoUI jUOS UOISNOUT.p say]Iqeqoid so] 
anb o1flusis e[9g “| op sprod un ndai sno} jUO sayqisstuIpeUT 
saseusul 379 Sa] 19 VWdd.J &@ npuodar jure saseusur 
SLT I SX] ‘aouanbgsuos us soispuod 939 yUO sgUUOT|WUeYOS 
919 JUoIeAe Inb xndd 39 9300 Op SassIe] a}9 JUO souUOTTN 
-ULYO9 UOU So[qISsIuIpe sasdeusul Say ‘sasAyeue sa] sueq “S// 
[[1e} op sojduns soiojeg[e suUOT]WURYOS XNap 911) SUOAR snoOU 
‘IO-xnood TMUeg “So[qIsstupe jUusIe}a saseusWI [/] 9 ‘Ino} 
uq ‘olpeyew oun .nb sajja} ‘suosie1 saqne,p mod npuoda 
sed juorwae.u mb sadeusur sop soidne IAins un oney op 
onbiyj9,| @ aeQuod s8nl 9}9 ke |] Jodrorpred ap asngar yuaTeAe 
inb xnod 3a JORJUOD op asud no sed jpreae A.u [I sfanbsoy 


6002 uIN! ‘ajanbua.p senbiuyoa | 


d9AB xNIOd ‘Saye 919 sed yuaTeAR .U IND sadeUdUT So] TAINS ne 
SO[QISSTWIpe JUdTe}q ‘asuodal-uOU Fk] ap dsned e] op UOTOUOJ 
ud soyyeys 939 juo sjuepuoda1 uou saseusul say ‘npuodar 
yuo ¢/Z [| Sjenbsa] tuued ‘saseusw p/0 8] =u ap ues 
Vda.l 9p aoyid ajygnbua,| op uojjnueyse,| ap ay[tey eT 


“asuodel-uou 12| soide oulewas 


‘“asuodai-uou 

ap Sed UN SUTWIOD 93181} 

919 ye [1,nb soide ourewias 
aun 919e]U0S adeUudjyy 
‘Q1eSIOATUUe 

uleyooid np spoyjsur e] 
uojas ‘aseusu ied auuosiad 
aun Jed 1jduwias sl1euuonsand 
(OVI 

yo Joided aseuuonsanb 
‘JauIO}U]) 3x1 9pow ke 
SdQUUOP SdP 9}99]]09 ap ue]|g 
“SaqNUIWW sIoy e aun 

ua dipuodai ynad uo jenbne 
129 Vdd.] 2p sao suonsanb 
Sa] JUeUD]UOD 9sUapUod 


dUN assoIpe | ap UOTIeIDAIFV 
“INIMAITAIOIUL | 

e dA0AUd JUepUods UOU 
aseusul np sanbijsiia}oRvse9 
sop sutidun surnsay 

‘auld oun I10A9901 

ap a1qudaosns inaMatAIO}U] 
“UONRIIOUI Pp SoINsawW sap 
ILJO & 9S1IO]Ne INIMAITAIOIU] 
‘sIOW xNap ap onpuajs 

UIR1ID} 9] INS [1RAPI] OP Bpol9g 
“SINIMITAIOUI 

xne dp1ajJjo ayiod e] ap 

sed ne u0loe19qul | JUeUIZDU0D 
airejusuajddns uoneul0 j 
‘asuodai-uou e] ndol eae inb 
IN][99 Op JUIIIFJIP INIMAIAIOU] 
"SJe][NSOI SINd] [IW 

Sd] JUdTeUDIQO ‘sjuapadajuRe 

Sa] UOTaS ‘INb sINaMaTAIOIUI 
so] tuued yuswanbiydess003 
SQUUOIJIIIIS SINIMIIAIOIUI QZ 
‘OVdl 

Jed a8eusw np saiquiawi $3] 
sno} sipuod9i yuateaap jonbne 


JUIUWIDLOJ JITBUUOT]SIN) Vd4.] op aileuuonsan?) e 


aseq ap suolsanb sap sys0iddy jeddea np sysoiddy 


IAINS 8p 9pnjd,| 9p saydoidde xnap sap sanbysiigjaeV4eD 
Z nes[qeL 


‘QUBI] Ud aTeUUONSanb 9] no suTdut 
omeuuoyjsonb a mjdwol op saiqiy jusreja sjuepuodal 
SY] ‘OUBI] Ud sITeUUONSaNb 9] JULUD}UOD YSLINIAS JOUII}UT 91S 
UN SdAB UOTBOTUNUIUOD Ud 99.)Ud,p sINpsd0Id Fe] 19 sUTUdUTT 
aleuuonsonb un ‘sourae,| e sdAOAUa UOTRJUASaId ap 3179] 
aun ndor yuo atjqnd suoydsja} ap o1sumnu ap sed jueke.u 
xnay ‘ajqnd suoydsja} ap orswinu un jyuepessod saeseusul 
S23] SNO} Mod dasijyN 939 & DIOTWIAp M9 ‘OV IL] UoNeoydde 
aun jo Joided uoneordde oun ‘ousty ua uoneordde oun 
yiepoduros 9yxTwW apoul e ued aT ‘a}XxTUI 9}99][09 op spoul 
dP SUOT]IPUOD sap sURpP addIge dIIeUUONSoNb uN gssI]IyN sUOAR 
snou ‘aseq ap suonsanb sap syooidde.; suep onb sipur} 
‘OVd] Jed ansturupe soseuswl soy jURSIA [eUISUO olIeU 
-uoysanb a] asAojdura suoae snou ‘jaddei np oysoidde | sueq 
(L007) UaNoYDS 12 UAggoD Fk 19 (LODZ) USINOYDS B IM9}99] 
3] SUOAOAUAI snoU ‘suOTsIO9Id op snjd Mog “7 neo[qe} ne 
sagjuasaid yuos Wq47,] & saanbijdde aseq ap suonsanb sap jo 
jedde1 np saysoidde sap sanbysuayoeses soyediound so] 

(p00) 
J3O0A ep 19 (SOOT) dooig sp sapmys sep jURULUIS sUONep 
-UPWIUIODAI sap ajdui0d nud} SUOAR snoU ‘TAINS op 9pnjd.] 


sojenbua xne esuodai ej ap ayiAeyUuasgidai e] ap sunayeo!pu] : WaYajUujeg }9 UBqgOD ‘Us}NOYoS 


angojeye9 ne X-[00-ZI .N ‘epeued enbnsneis 


MOAVIUOD IMOg ‘31q0}00 ke jaTTIN ap Wdqq,] Inod yoRU0 
ap osud no sed year Au [I sjanbsaj dace no sagyrey 939 
sed jusreav.U sasuodai sa] juop ‘radioed ap asnjar yuateae 
mb Vdq,| & Jedtonied mod sauuonossjas sadeusul ap sauLoj 
juate}9 suol[HueYyos Say] ‘anne| suep (pg6l WoysTyiog 
}9 Ud]SIOy) Seq op suoNsonb sap asyooidde., uojas 
Jo S¥O UN SUep (9p6| ZIIMIMIP 19 Udsue}) Jodder np syooid 
-de.| uojes ‘yqq.] & sjuepuodsi-uou ap suoj[nueyse xnap 
ap Soidne MOIAINUI.P SANLIUd} BJ2ANOU oUN UD a\SISUOD 
eB opms,J ‘seg-skeg sop (Wqq) 2an0e uoNeindod e] ms 
s}gnbuq | 8 asuodal-uoU ap sed sap a]]oYO9 opueIS B TATNS UN 
gst[ea e seg-sheg anbusneys ‘¢Qoz a1quiacap ke yarfIn{ aq 


S007 9P 
TAINS ap apnjyy — 9Ade UOLE[Ndod ev Ans sanbuq 7's 


‘plepuL}s [VULIOU IO] Be] 9p 10 — | aHUeNb J 3s9 ae, no 


(¢7) CaS <2 FY) = 4gld 
wos “(€Z) red 
saguUOp SddUIysa sadAj-sinaua say yueAo]durs ud jo aTeULIOU 
JUSWIDAT}eWUTXOIdde ysa_Y ap Io] eT onb yuesoddns ua %% (7 
— [)OOT d0ueTyUOS op SoyJeATOqUT Sd] SUOUTULIZ}ap SNON 
‘007 = ¥ MIA P| Sap 9BIOAUOd 
yweae odAj-Inou9,] ap uoTeUsa,, onb sed saj sno} suep 
9]8]SUOD SUOAR sTeU ‘QQS = g eB nbsnf jure g ap sinayea 
sopueis snjd ap asyuaurtiadxa suoAe SNON ‘sopnyd sa] soyno} 
Inod 007 = g suouaid snou ‘suonewrxoidde say sueq 
“QUIIISO USAOU Y INdVOIPUT | 1S9 wen rea T= rad 20 


Sin baal 
-1S9,9 “Y Inayeorpul | ap odAj-naU9,| op UOTFeUT}Sa UN 4s9 
sonbyda1 g soo ap anbundure toy ve] mod odAj-moua,J “g 
SRE [0g sees $y Inayeorpur,.] op sonbijda1 g suop 
suoua}qgo snoN “g deysjoog uoj[WueYse onbeyo snod gynoyeo 
}SO Y INoeoIpul, J ‘ajquiasua jad onb wu oaTIe] ouIOW op 
}2 So[BUIZLIO saquUOp ap s]qQUIASUa | Op INed eB ISTUIOI DOA 
jo JUaUTUIepUsdapuT so} sUOT[MURYDI,p 113e,S [] ‘dens}ooq 
suoyyueyoe.p g “°° ‘7 ‘. = q siquiou un jUeUUOTSaIIsS 
Ud dOUIT}S9 }Sd Y IMajeorpul,| op odAj-InoU9,| ‘opoujow 
ayeo uojeas ‘(€66] TURITYSqI] 39 UOoY”W) onbijowresed 
uou deysjoog np apoyjsw sun sed y Inayeorput.| ap odAq 
-INdL9 | SUOWT}SA SNON “odAj-INdU9 | 9p SoAreu sonbuiowmMuU 
suoneunxoidde sap Jasijyn e suoUsISoI snou snou ‘Ido] 
‘gsuodai ap sayyiqeqoid sap uoljeun}so,| op asned eB o[dumts 
sed jso.u y ap odAj-moauay ap onbndyeue [noes ug 
; ‘omeyuoua|ddns uorstsaid 
ap oyiad oun asnesd inb 39 “uo][HUeYd9,| JURSITIN Ud SUOUITSS 


OLL 


andojeye9 ne X-[00-Z1 .N ‘epeueD onbnsneis 


Sa] snou ‘sayja91 asuodai ap sayiqeqoid soy sed suossteu 
-u0d dU snou sUTWIOD ‘uoTSIO9Id op aod ouleuso oun 
yeloutequa ‘ouop jo ‘uonejndod ap y IMajeorpur.| op jofns 
ne spmyipsour oun swiguI puenb jeiMmpoyUl uoT[HUeYsE uN .p 
a8vII} a] ‘Sayjao1 asuoda1 ap sayt{iqeqoid say suorIssteuuod 
snou I§ ‘soguuoT]yueYysa sayun sop asuods1 op osuUIsTU 
-PO9UI J] ‘puodas 9g] Jd ‘uoNeindod ev ap sseuUOT[NURYDE | 
\sd Jotuaid a] ‘naf ua juaqua sallojeaye snssad0id xnop 
Ip juswianny ‘sosumsa asuodai ap sayyiqeqoid sap uo] 
-[Hueyog,p odAj-ye9g,| MUSAIOIUT Wey Y Y IMoyeorpul,T 

"sadAj-sinde snd] ‘Toumsa SUOASPp snou 
‘SagUUOP Sap 9}99]]09 op Sar3a}v.ys NO sajgnbua sas1aAtp nod 
Y Inaeorpul,| ap smayea sa] Joreduiod suojnoa snou IS 


JIULIFUOD ap J][eAj9}UT 39 OdAJ-ANIAAW: OS 


‘sodA}-sinauie sap uorneurxoidde 
dUN SUOUd}GO SNOU JUSUIWIOD SUOALID9p snoU ‘[°¢ UOT}OaS 
B] B “PIO JURAY “UlLID} JJ ANS [IVA Op SaIdayeNS SayUdIII FIP 
sINd] op sANeUasoida1 ey e Wey e Inb 90 ua said snd 
ap sopnye sdd suoUTUIeXd snou ‘ES 139 ZG SUOT}OIS xNY 

‘seg-skeg sap (uojluopy 
Ajafpg JDUOIDON) DUNdBS ke] Op JOURT[IOAIMS Op syeuOT}eU 
ajonbua | op sped a] suep sonbijdde a}x1uw spo e saauuOp 
ap a}0a]]00 op surjd say oWId0UOD SUIaIxNAp eT (HP86I 
WdYyIyI9q 19 Ud}SIay) saseq ap suoNsonb sap ajo ap 3a 
(9p6[ ZIMINnY Jo uasue}) Joddei np syooidde.| op uouiexa 
un pudidwios aq ‘seg-skeg sop (Wdq) eANoe UONeindod 
eR] Ins ajonbuq,] e wen e oiomuoid ey] ‘saguUOp sap 
3199][09 SP SAPOU SP SUOSTBUTQUIOD SdSIOAIP Jo asuOdsI-UOU 
dP Sed SOP IAINS ap Satsa}es SASIOAIp Ins URLIOd sapnje sap 
B Y IMojyeorpur., suonbrjdde snou ‘uorses ajuesaud ey VW 


UY Anojwsipur| ap uoyvoyddy -¢ 


8661 9P STOd 
ajygnbua | suep }d¥}U09 ap astid ap saanejua} 
saigtwaid xis say inod jewixew njosqe siveig p ainsiy 


asuoddl op xney, 
01 8°0 9°0 r'0 av 00 


SLL 


‘0% Q OP INOINL MI[A JUN SIIA 9BIOAUOD & Y 
INILOIPUL, | ‘AT}LJUD} SUIIISION eR] Sady ‘joRJUOS ap asud op 
SOAT]EJUD} XIS Soy INod asuodsa1 ap xnv} Np UOTIUO} US gJUAS 
-91da1 4s (d)“g [BUIXeUT Nose sieIq 9] “p aNd el V 


s0‘o = 4 19200 = 4 ‘10 =4 
Ins sagseq }UOS SaiNatigJUl sautOg sa] 
8661 9P S7TOd 2}enbua jueinp 3dej}uU09 
ap asiid ap saaijzejua} saigtmaid xIs say inod 
Y Y ANILIIpul,| ap sainaigjur sausog =  ¢ ainsiy 


asuodal op xney 


0'l 80 9°0 v0 70 00 

¥ S . = - + Tale S 
iS 
S 
PN 
° 
me 
o FW 
mon 

eed 

aus = S 

owen ramen Ct he 

se 


"0% ¢ Op NedATU 
JJ Ins s9puoj sd INb oIMdLIgJUI sWIOg ane] e Ssonatiadns 
stewief yUOS OU SINa{eA sa] ‘Juepuadag -% cs‘, ap nedArU 
np syooid jsd Y IMayvorpur.| ‘saanejusa} oyenb saidy -% O| 
ap neoAlu ne jURpuodsauiod aMdLIQJUI SUIOg kB] B SOINIL 
-odns jUOS Y INIBOIPUT.| Op SIMayea say ‘joR}UOD ap asud 
op SAT}RJUd} OWITxNep eR] Ssoide.nb onbipur ¢ smsy ey 
‘600 = 4209 6400 = 4 ‘10 = A Nos ‘A ap smayea sIon 
ISIOYD SUOAR SNON ‘8661 2P STOd aHenbus,| yuemp joR}UOD 
ap asud ap sdanej}ud} xIs soy nod Yy SdArlasqgo Y SIMayeoIpUul 
sap yo (d ‘A)'4 asuodai ey] ap auaneyuasaidar op UOT}OUO] vy] 
JUIUOD € dINSIJ BT “¢°¢€ 39 [7 SUOTIDS xB 9sT]IN a] dulaxd | 
op UdAOU Ne UOTeSITPULIOU eT NBIANOU dp SUOISNI]I SNON 


qjduraxq  ¢"E"b 


"§ UOI}DAS PT B Sa}LID9p Sapnye say Inod suojnoyeo 
S3] SNON ‘asuodsi ey ap ayANeUasoida1 op suonNoUOC} ap 
wiou a] (d ‘A)% ya (d ‘A)’ B suouUOp snou ‘Ins Inb UOTDes 
BI sued “Y INdedIpUI,] Sp saInotgzut sautog say (d ‘A)% 
yo (d ‘A)'&u sed suoyuassidar snou ‘(7Z) 19 (QZ) sueq 


Cea) A) Sata se Jhkdp esi 


® SUOALU® SNOU 


6002 unl ‘ajyeanbua.p sanbiuyoo) 


(12) ‘AS (d)"F 
onb 
JULpULUSp Ud Jd (g]) JURSTTNN Us ‘onsoyeue soTUeU aq 
(02) (J Ayu = Adp-1 ZY 
anb ay9a ang Op |! 
(61) ‘AS (d)"g 


onb suopuewiap 

snou 1s onb ‘“(/][) op yueyed ua ‘1anUOUI ap a[I9RJ 389 IT] 
‘$0 = Mg 
Jenbay mod onbnsyjoddy wiayt un e neaAnou ap suojTUT 
snou snou ‘uanbgsuos reg ‘ajgnbus.p widyI,] ap yuapuadap 

Y IMaed1put | ap sammauayut sauioq say anb yuaptag js9 |] 
‘qUepuodsaod sug FI B InaLIadns ane Wop 
Y Moayeorpur,| ‘mMayea sutepyao oun apeoxa suuaAour anby 
-eipenb mad] ap s9ed dUIDeI RP] NO asuodaI-UOU ap sIeIq 
3] onb sed ynda ou IdTUWIAap 99 Ig “soguUOP $990 ap IMayesT]NN | 
Jed ajgnbua,p sasuuop xne sagsodum oyyenb op sajyurey 
-U09 Sap S}JUSIPsISUI sap UN,] JUe}D SUTWIOD dISIOYD ane 
ynad SIMILISJUT SUIOG a}9d) ‘a}UIDsaId IMa]eA 9UN eB IMILIJUI 
ylos suusAow onbyesipenb mo1ud,] op saued UTD] eT NO 
asuodal-uou ap njosqe steiq a] onb yuepueWap ua Y Ima}eo 
-IpUI_.] Op aMaLaJUI sUIOg 9UN M]QeI9 SUOCANOd snouU ‘USUI 
-dSIOAU] “Sasuods1 sap (aaysnfe) suudaXhoul ke] ap suUaAOUI 
onbyvipenb maa] op seed suloel ek] ap ja osuodal 
-UOU op sIeIq np samouedns sawiog sa] JouTUIajap Imod 

UY Mayeorpur,| gstjyN suoae snou ‘oyuapgogid uonseas ve] VY 


gsuodg.s B] 9p 9ANLJUISIAdI1 Bp SUOTIUO 7 E'p 


“ste1q un Iayuasoid 
op sojqudsosns yuos nb ja uorstda1d ouTeys9 oun jUspessod 
Inb samoyegje sajqeuea sap neaanou ap juos (gT) 39 (ZT) 
aonb IausI[NOS SUOADP SNON “¢ UOTIS FR] B SaILID9p sapnjad 
Sd] S3}jNo} op sed oI Suep (QT) 19 (ZI) SUOTNITeD snOoN 


(81) (MD) 5 + Def = 9) "a 
Nipecigh gaat 
(LT) (@y-D (9) ¢ 


B Sayedo JOS SaT]q 

‘(d)“q 12 (d)“g sed sayuepuodsaii0s saulog sa] suoUsISSp 

SON ic ‘Ov= (4 )s onb suosoddns snon ‘anbnamoddy 

ajonbus,p wid}t un Tulyjap op spourwios jusnbssuos sed 

}S9 ]I ‘suosiereduios say Mog ‘ajenbua,j ap 4 wrlay1 onbeys 
mod sayuslayip yuos (4 ‘d) “dT yo (4 ‘d) “d SOUIOg SIT 

(4 ‘d) "7 

ed sagusisop ‘saaumysa asuoda1 ap sayypiqeqoid sa] atojdwio 

mb uoj[HueYyd9,| Ins 9puojJ MayeuINsd,| SUOSITHN snou ‘aoe]d 
es V ‘(4 ‘d)“gq sed suossteuuos ou snou ‘neaanou 9q 


ansojeyed ne X-[00-ZI .N ‘epeueg onbysneis 


(91) (4 ‘d)"q = 
Cd pone aig hus 
(Ha) s + CHD) gh = (HOwWOra 


onb juswaanjeurtxoidde ayaa ysa tT “44 ap (WOdy) 
suushour onbyeipenb Imeua,] ap sai guided eR] Imod 
sWI0g oun I[qQeuIa SUOANOd snoU ‘aIZTURUT sWIQUI kT 9G 
‘(4 ‘d)“g red mayeurnsa | suousIsap snON “sdsuIysa 
asuodai ap samiqeqord S93] 19 UOT[MURYSS | JURSI[QN Us JOW 
-1]S9,] suoAnod snou sieul ‘(¢{) suep (4 ‘d)“g amauodns 
sw0g ke] sed suossteuu0d ou snou 4uUdUIa\sazTUR/\ 


(SI) ‘(4 ‘d)"g = 
dz d 
sty) Ms@s >" 9a! 
oe ap 


(p[) sterq 3] od sauiog ud ej99 sIINpey SUOANOd sNON ‘s2]q 
-elieA xnop sop sadd}-syed9 sap yinpoid 9] red njosqe suas 
ne 990g so sonbuosjenb sayqeueA xnop oud soURLIRA 
-09 eR] aonb s0u0US a]eSouI aed “Y op UOHesTTeWUOU 
oun IJ3ANo0y op jyouued zremyos-Ayoneyd op syyesoul.T 
(8861) WeysTyIag ¥ SUOIaJOI snou snou “4 ap (‘d) _s 
soueueA ke] Op a0N9 uoNewlxoidde oun mog -gsuodai 
ap sayyiqeqoid sa] 39 SoIpMjs SWId} Soy aNUS UOTe[Ndod ap 
soueuRAos ey (d — 'd) (4 — “)K n/t =(d “)9 ooae 


ge Thic= Sle 
(v1) Eas ("Og 


B eso JUsWOANeUUTXOIdde ysa_ (+L) g sterq uos anb (¢00Z 
WOQspuNy 39 [epurEsS ‘“QR6] WoaystyIeg ‘ajduioxe Jed) Joy 
-uoul jnad ug ‘ajenbua,| e sasuodai say ins saseq uonrindod 
ap suusXouwl ve] ap uosduioy-ZIAJOY{ IMayeuysa,, A jos 
32 aJonbus oun suep soinsaui jsa Inb ajqeueA ouN { HOS 

“soueugos sorld sa] snos 39}J9,] JonUOW mod Y mMo9}edIpUur | 
B® saulog sao suonbiydde snon ‘sasuods1 sap sagysnife 
souushour sop (IWWOWY) euueAow onbneipenb mou] 
ap salvo ouTORI e] & 19 DsUOdaI-UOU ap sIeIq ne sammotiodns 
saulog sap Jasodun mod asijqyn ano jnod Y mMoyeoIput | 
‘X aenbus oun,p wo} yno} mod ‘onb suonuow snon 


suusAow onbyeipenb 1n3113,] 
IP 99AAVI IUIDEA Jd [VUNIXBU NjOsqe sivlg ['¢'p 
Ye 
yuepuodsau0d Y IMdjedIpUl.| Op UOTJVST[eULIOU e| suomosIp 
SNON ‘sajgnbua sapuess say od juan asaujodAy ay99 ‘steiq 
sues oipuodai e suorsuadoid say 19umNsa suoanod snou no 
g[evapl UONeNyIs ke] SUeP SUCANO] snou snou aonb suosoddns 
snou ‘tor ‘uonbasuod seg “Y IMoeOIpUl., sp UONest] 
-PULIOU B] IMS dOUapPIOUT 9UN JUaW[eS9 yUO sJUapsogid UO} 
-998 B] & SZAgTNOS UONRIdINUI,p sous]qold sa] ‘aouapIAg 


sajenbua xne esuodai e| ap s}IANe}UaSsaidai E| ap SuNayedipu| : WaYa|yjeg Je UBqqoD ‘ue}]NnOYyoS PLL 


an3o]e}e9 ne X-100-Z1 WN ‘epeued onbysneis 


31nNo} oq ‘UOHRoyIUsIs oun uae “Ma[BA eS Op SUOTIRULA 
Sa] ‘ouOp 32 ‘a]Jayde UOs onb ue Y Ina}eoIpur UN ke SaIOg 
sap Jauuop MoAnod suojnoA snoN ‘uOnesITRULIOU ke] Isa 
Y IMajeorput un _p aueyodun snbysugioVsed SUISISION eT 


UOHLSTTEULION §¢'p 


‘uorstoaid eB] Ans onb sure sieiq 3] Ins soUaprloUl oun B 
UOT[HULYE,| Op o]Ie1 ek] ‘ANNO UY “Ja]Nd]ed say mod sogsi[NN 
319 JUO Inb samerjrxne sayqeuea sop jusuTUTepuadapuT sor9p 
-Isuod ang sed jyuaanod ou s]j ‘sousprid ooaev Y SIMo}VOIpuT 
Sa] Jasin snofno} yey [I,nb jusNUOU suONeAIOSqO Sad) 

‘Sajqeieduioo uoj]HUeYI9, p saq]Ie} sop Inod 
surg ‘aydurexe sJarwiaid a] suep saystumdo snjd suotias snou 
‘quod ‘aATROTJIUSIS Jso Inb onbrun uoNNg UO. opuLIs eT] 
anb sayIoj Issne JUSIOS Saj[a ‘aTqudsua Nb steUul “SATVOTJIUSIS 
JOS dU UOTINQLYUOS ayyH0d suNONe ‘JUOWT[aNprtAIput nb ynod 
9S |] ‘opueIs JOS UOTNGUUOD es onb stew ‘aUUdIAIOJUT UOT 
-OvIO}UI ayNds duN ‘asepuos one uN suep onb spur} ‘sofqiey 
SO} JUSIOS Sd][ONPIAIPUT SUOT|NGLyAUOD say aonb sreur ‘ampuodal 
e suotsuadoid sap uonotpaid ey e jUaNqLnUOS soruNeI suOT} 
-OvIO}UI.p sIquIOU pues UN ‘asepuos UN suep ‘onb yIeuMod 
as |] ‘a]QQns uooUTsIp oyNe oUN IOI JITe} SUOADP SNON 

“MAY UIsSoIdaI 
eR] ap aystumdo snjd ona oun e oUOp elNpUuod aI} 
ayjod ap uoy]Hueyss un ‘Je1auas uq ‘oIpuodal ev suotsuadoid 
Sa] Jowmnsa,p sed juowia;duns 3no} yuayauLIed ou suoTHUeYIeE 
sjjed say “[ ap suIMso Y INayeOIpUT UN 3a SPIA s[Ipou 
un asste] Inb 30 ‘aajda00r elas ou samerixne sonbijsiiajoese9 
Sa] 19 asuodal ap yUatd}IOdWIOD 9] ANUS UOTIORID}UT SUNONe 
‘uoyTHuRYSg Jed Is un Mog “OT [le ep uoTTHUeYS? uN,p 
SWIINXd UOENIIS LI AI9PISUOD UO | IS AIPUdIACUIOD & d]IDRI ISO 
juIod 3¢ “Y INd}edIpuT Jno} op UOTJeUIT}Sd,.| SUBP OURLIBA UN 
J2 sIeIq UN oIINpoQuI jned sjapour np uOTeIas ek] ‘a}UeINOD 
anbysyejs onbyeid oun jso uoNeousIs ap nedatu uN IMs 
JULPUO} 9S Ud SoTIpow op yUdUTA}snle |] IS DUI “SAQTP 189 
SOATJLOIJIUBIS JULIO SUTUIOD sadj}da50B SUOTIORIDJUT,p sIQquIOU 
3] snyd ‘puess isa uo]THUeYS | snyd £ UOTTHULYDS,] Op aT]1e} P| 
op puodap orpuodai ek suotsuadoid sap uoneunysa | nod ajap 
-OW 9] ‘Sed SWOTXNap I] sue] ‘JUSUTa}snfeins un Jed 993995 
-je ong weunod add}-moauod,] stew ‘steiq unone neaAnou 
9p suolMpoyuI.u snou ‘seo Jawad af sueq ‘Iuyapoid 
nesatu un e yodder ied saneoyiusis }sa uolNnquUOS 
B] JUOP SoTQeLURAOD Sop sipuddap sJapoul 2] Jassie] suoAnod 
snou no gfqefeaid ne sajqeueAod sa] JueXxI{ ua aipuodai 
B suotsuodoid soy] 1auINsa nod ajapour un Jasodum suoanod 
SnON ‘suondo xnoap 9ajsIxd ]] ‘aJapour np jusurajsnie,] 
aP 19 UONDIIIS PR] Sp 9sned k ‘Sa]UdI]JIP STOJa}N0} JUOs asuod 
-91 9p soayIqeqoid sop uoNRUIsd,| Ins saouaproul say 
‘sjomjouod simajeuwjss so[duts ap nary ne soueyuos 
op Sa][BAJOJUI sap juesITyN us adwioo nua} ang yWexumod 
[I JUOp UoTsIO9Id s[qIe} oUN JUdIeMe sIMa}eUISa say ‘UOTTO 
-UBYO9_,P SaT[Ie} Sayed so] Inod ‘ouog ‘steiq sues uone;ndod 


ebb 


ap Y IMayworpul.] JowWNsd,p soajqeded suouias snON ‘sIeIq 
ap sed jresmnpoid ou jo soueLeA oun e nb nol] yeJouUOp ou Y 
INILOIPUL | SP UOT[HULYSI,] INS s9pUoJ UONPUIT}Sa | “SoTjonp 
-IAIpul aipuodal e suotsusdoid say] suOIsSsIeUUOD snoU IS 
“UOT[HULYOS UN URSIN Ud Y IMIJedIPUL | IP d]][99 39 
aipuodar e suotsuadoid sap UoeUMsSa | dIp-e-}S9,9 “UOTRUI 
-1]Sa,p sodej}a xnap Sa] Ins dUSpIOUI 9uN k aI[q ‘oOpuBIS JUOU 
-arreniqie sed jso.u uo]nueyse p aie) ey ‘onbneid uq 
"S}UdIOYIp a9 JUdAnad (¢) 39 
(€[) SMMoyeoIpUT soy ‘WUSWAT[OIMeN “osuOdaI ap soyT]Iqeqoid 
Sa] Joumjsa mod sasijqyn juos ‘olleoul] UOISsoIso1 e] NO 
onbysi3o] UOISsaIB91 B] OUIUIOS ‘prlepur}s so[spowl sop puenb 


Pe oe ee arpa: 
(ct) ‘(d- DN Tose. 
H 


OP JUIBIDAUOS INd}eUIT}sa 
oun jso y onb Januow ap o[lo1gIp sed yso.u [I ‘y ayens 
eB] suep asuodar ap saytqeqoid sap uonendod ap suuaXou 
ey 4d red jo y ayens ef op ayyiey ey “ay sed ‘yx sed sommyop 
saves soy H ‘°° ‘Z ‘T = y sed neaanou ap suousIsap snou 
IS (YX op sasse[o soy Jed stuyap xnoo aonb uonejndod ey ap 
sadnois-snos saqnne,p ious, e assuoda1 ap say{iqeqoid 
Sd] aNUd sadUdIOJJIp soy sed djd]Jo1 OU Y IMa}KOIpUT,T 

“Q[qQIey SIANLJUOSoIdaI ey e Woddel Jed pre99 UN sIO]e 
dINSOUI Y INdeoIput, J ‘sayqruodsip ayo JsTe1Aop sonbyuapt 
sa[quiasua sop Yuepuaday ‘ajapour np sayeorydxe souessind 
eB] antoisoe sed ou yuoAnod sayja,nbstnd ‘arpuodai e suors 
-uodoid sap uoneumnsa,| mod samerixne sayqeueAa say soyno} 
Jasl]yn,p oiessaoou sed jso,u [I,nb JousI[nNos ap JUSIAUOD I] 
"SOWIQUI S3] 99 JUSATOP SaTqeLIRA Op SoTqUIASUd Say ‘SUOSTeI 
-eduiod sap mod asiyyn 3so Y MIyeoIpUl,] IS “SoTJONprAIput 
aipuods1 e suorsuadoid say JowNsa mod oa]Us op IAJas 
UO Inb sajqeLieAod ap a quiasud,| IMa]eA es ap jUdUTa|dWIOD 
SUIWUOD JasI[yN SUOASP snou “Y INs}edIpul UN suOSsT]WN snou 
anb stoy anbeys ‘anb amp jnoa ejad ‘sallelIxne sojqeuea 
op squiosua,| e Woddel sed s{qrey aAyRJUISaIdal eT op jofns 
ne.nb suoisn[auod sop 101 suoAnod ou snou ‘ouo0q ‘sayqru 
-odsip soulel[ixne sajqeuea sop « UOTNOSEI op OAnod » np 
ejap-ne sipuodai e suoisuadoid say Joumsa,p sayqedeo sed 
SoUUOS dU snoU ‘ajapour janb spoduitu rajsnfe e@ SUOALITe 
SNOU NO Sed J] SUBP SWIQJY ‘a[QIeJ 39 BWOJ dIANeJUSSOIdI 
ve] luyjap suoae snou ‘z uonoss ve] YW “wed ayne.p 
‘sourel[ixne sojqeueA op uonesiyyn.] 310 aqrjiqruodsrp PR] 30 
‘ued oun p Y IMd}LdIPUT | IUD UOTLIII 9UOJ 9UN 9}STXO T] 

“SOTIVI[IXNE S3TQeLIVA Op dXxI} B[quUasUa 
NO} eB JJapouwl jno} Jaysnfe.p sajqedeo soumwos snou ‘yIp 
JUNWIaANY “aw9]qQo1d ap sed asod ou aipuodai e suorsuadoid 
sg] Inod s[9pow uN.p uoNdsJ9s ev] onb jo ajo uNone 
enof ou uorstooid ey onb ay0s op opuels juoWoMeNIqse 
so UOTTQURYOS,p ole} ev] onb psoqe,p suosoddnsg 
é [I-}-0umsa nb stepy ‘opnyyoexd outed oun apassod jI,nb 
}9 steiq un Ia}UasoI1d ynod [I.nb oaMp-e-jsa,9 “UOT]WURYDE | 


600z uinf ‘ajyjanbus,.p senbiuyoe | 


ap puodep [I Ip juowenny ‘alojyeaye ajqeuea oun Issne 
}S9 [I “IMo}VUISS UN SUIIWI-IN] Jsd Y INa\eorput,] onbsing 

"Y suMayeoIpul sap uosteedus0d ey 39 
uonejeidiayut | suJau0d Inb 99 ua sajuRyIOoduM san saouanb 
-9SU09 SOP JUO S}USWIZI9 xnop sad ‘a]janpraiput asuodar 
9p si[Iqeqoid op smajeull}sd sap Ins 39 sadepuos sap suOT]N 
-UBYO9 So] INS gpuoy yso nb Y Inajeorput un Inod majeur 
-11s9 un & sassed SOWIWIOS snON “WWornsoaur s]t.nb ydaouos 9] 
}9 IMI[VA IMI] 19}91d19}uT sUOANOd snou aJanbey] Dae 9yT]TORY 
RP] ISO Y SIMayeoIpul sap onbysusjoere~o ouleIxnap ey] 


uoneQI1d19}U, Zp 


‘JUdAINS Inb suOT|das xNap Sa] SURP 
juowaredgs saguruexa jUOs sonbysuag}oeIeos sayne xnap 
SIT “‘sIMs}eUIIss sop e jURSsed Ud SaTqeINsoU Y SMoajeVoIpuT 
S3] NpUsI suoAe snou ‘oUDG ‘asepuOS UN ke sasuodal sap 
anb asodstp ou uo] aonb 3a sanuuooul yuos ampuodal e suoIS 
-uodoid say puenb sajqeinsow sed juos ou nb « uonejndod 
op » Y SINaJeoIpur sa] Inod ssuodad ek] Ins sapuoy sInayeusa 
sop 9Aulap elop suoae snou ‘7'¢ uOTdaS BF] YW “Y sINayeo 
-IPUI So] JostyeuuoU 39 Ia}9Id19}UI “IamMsaul moanod SUO[NOA 
SnouU ‘90UR}SIP Sp UOTOUOJ 9UN 99Ae UOTeIAI sun Rd VY 

“UOeICI9}UT | STIOeJ Qour)sIp 
op uonouoy ey, ‘auusAouwl sspuodsi e uorsuadoid ev] ap 
MIVA KI SUOXIJ SnoU Is ‘sToyaynoy ‘aoeydap as yutod 90 ‘seo 
sop uednyd ey suep ‘sayjonprarput suotsusdoid sap 3majooA 
3] SUOTJIpOW snou Ig ‘sayesa yUOS aIpuodal e suortsuadold 
S93] $dJNO} NO INJId MOAeRS & “JoI[NOTed yuIod un e yodde1 
Jed yre09 un oInsour Y IMayeorput un.nb sipur) ‘sonbuysut 
-AS JUOS SOIZTWIAP $d “d0URISIP Ap SUOTIDUO}J Soy 19 Y Sino} 
-BOIPUI Sd] BUD JTQNs UOTOUNSIp oun aIIe] SUOADP SNON 

“SQUSSIIDIUI UOSITeULIOU ap sanbiysLI9}9e1e9 Sap yO 
QUUSIPI[ONI SULIOU ke] INS sapuoj Y sINayeoIpul say aonb suoy 
-UOUI snou ‘¢'p UOT}DES FJ B ‘SIOJaINOT, ‘aoURISIP 9p SUOT}OUOJ 
soqne p e JUsTeI9UNUI snou ‘dns suLIOU No ‘tuNWaIdns suOU 
R] SUIWIOS ‘sowIOU sayne.q ‘aIpuodsai e suortsuadoid sap 
adAj-1e99,] ins otndde,s nb yY Inayeorpul uN e aUaW snou 
Iq Woururemos sasijyn aJao Jsa nb ouuartprjons suuOU 
R] SUOSSISIOYD snou ‘¢ UODaS BT Y ‘onbynewaYeU sULIOU 
9]NO} Sp JUSUIOTIORJ DdALIOp ane inad J0uR}sSIP 9p UOTOUOJ 
su) ‘UonEIdI9}uI,] ay[1I9ey Inb aipuodai e suotsuadoad soy 
suep UONeLIeA ke] op janied yuaswasseyo un jouLIEd souR}sIpP 
op wOTOUoJ oUN.p sueNsueLy aYesour.p goudoid ey 
‘onbyewayjeul suas Ne VOUeISIP ap oINSoUI dUN NO UOTOUOJ 
dUN INS S9PUOj JUSIOS Y SINd}eoIpuT say onb suojNoA snoN 


sayviguas sanbysiigjyaeViey =p 


‘asuodal-uou ap steiq a] nod ayduns ainauedns aus0q 
aun 31u3}qg0,p jowued gsodoid y moaqjeorpul., onb suon 
-UOU SNON “Y SINayVoIIpul sap saayleynos sonbysugyoeles 
Sap suoyNosIp snou ‘Uodes aUasoId ek] Y “UONPIOOSse.p 
onbueul 9] ‘saaueAnyNUr suOIpUod sop suep ‘IoMsoul op 


andojejeo ne X-[00-ZI1 .N ‘epeued anbysyeis 


Jnoal{qo mod yuo sMoayedIpul sao UaUIaT[aTUASs_ “sUOJ 3S9_Y 
SINDJLOIPUT SI] 19 UONLIDOSSE Pp SAINSOUI SI] 9QUS UOTIAI eT 
(8861) PlewoqoW] 19 eTTeg “YsIEW 19 (0661) JepUe “(6L61) 
[exsnry 39 URUIPOON ‘o[dutoxe Jed ‘xnoiquiou yuos uoenb 
-9PB.P SOIPUI SaT NO UONRIOOSse.p SaINsoW say ‘sayne,p 
aInysuod ua.p s[qissod jso [I ‘juepuaday ‘aytAye}Uas 
-g1dal e] 9p mmajeorpurl un suosodoad snou ‘¢ uosas e] VW 


¥Y Sanajvoipul sap sonbysiigae1wD = *p 


8661 
2P S1Od 239Nnbua,| suep }9e}U09 ap asiid ap 
SIAI}V}UI) SIAQTUIIId XIS Sa[ Inod y AnazeIIpuy =7 Vans 


asuodal op xney, 
01 8°0 9°0 70 70 00 


‘(pO0Z) UaINoYpS Jed saqtey 
SUOTIE]L}SUOD Sd] SULIIJUOD jeINSAI 9D) ‘aUANLJUSsSaIdal ey op 
UONNUILUIP UN d[IAdI INdJeOIpuT,] sTeuI ‘oJUdWUIBNe ssuoddI 
ap xne} 9] ‘omeyuawgtddns sanejua} onbeyd mog ‘dAnRjUI} 
aun saide syuepuodai ap s]quiasua,] e@ puodsa0s ayones 
® snjd ey mayea eT ‘STOd aenbusa | Jueimp 39e]U09 ap osud 
ap SdANeIUD} sarotwiaid xts say od asuoda1 op xne} np voy 
-oUOJ Ud yUdUONbIYdeIs gyUasoIdo1 Jso Y ‘Z BINS el V 
-snyd 
jo SUB CG Op ja SUR pS B CE Op ‘SUR PE BC AP : SAUOSHLI SION 
pusidwios a8e.p a[qeueA ey] ‘s9uUOTOa]aS sUUOSIOd ke] Op 
sjuared sap aouessteu ap sXed 9] Ins sapuo}j }so UONLOTJISSe[D 
ey] euapioco JeU-uoU ane jo [eJUSpI900 UOU JNeU-UOU 
one ‘siey[Wue-opurpJoaN “steureuLing “omy “ulesolRey] ‘Jeu 
: Sau08a}e0 ydas ua styedoi yuateya sonbruyja syuspsogjue 
SOJT ‘SOJOUNSIP saosaqvo sWIOD (WYDaN~) 10 BAe eT 
‘WeployVOY ‘Weplajsury) sapueis snjd soy saytA onenb soy 
}9 SouTAOId Z| Sa] MIP-k-}S9,9 ‘SaLIOZg}vd QT B UOTBOIJISSEIO 
aun 3Ie]9 UOIsd1 ey] ‘saUepUsdspuUl sarqeLIeA SUIWIOD 
a8e,] 19 sanbruyja sjuapsogque say ‘uoIso1 ev] yUeUsIdWIOD 
anbysiso] UOIssaIsd1 Op d[9pOW UN,pP SIAJOS SOUWIWIOS 
snou snou ‘asuods1 op xne, sop uoneumss | mod 
"S9QUUOP Sap 3}99]]09 ap opouied ey] yURIMp 
Y Inaevorpur.| ap aqimood ey] dAIns op stuued e snou mb 


sejenbue xne esuodail e| ep ayiAlje}UaSeudai e] Bp SunayedIpu| : WaYa|Uujeg Je UaqgoD ‘ua}noYyoS ZLL 


anZojeieo nev X-[00-ZI .N ‘epeueg onbusneis 


30 ‘asuodai oun eB WNoge HWeAR 39R}UOS op osud op sATRIUA} 
anbeyd Is jinpep 39 uUOT[HUeYD9,| B UleUId} 9] IMs [leary 
J] JURLIDOUOD SdATLIISIUTUIPe SOQUUOP Sop SI[OI SUOAR SNON] 
‘UOIIAUD %% (9 ap }Ie}a BsUOdaI Op xNe} J] 19 YOO Ob OP Sold 
ap Hel9 UOT[HURYDE,| ap aT[1e} eT Juowonbrun OV] dA} ap 
JUSIE}S SIOUI JoTUUdId NP SMOIAIOJUT So] o]Jonbe] suep sonbru 
-oydg]a} SMOIIAINUT,p Jd doRTd Ins SMOIAIOJUI.pP UOSTRUTG 
-WI09 9UN Ud jIe\SISUOD dJgnbud,| onb suojaddey *[°Z uoNses 
PR] & ALOSP 8661 OP STOd aenbus,| op yueusaoid saguuop 
sop e sasodoid y smoajeoriput say suonbydde snon 


ajduiaxy = ¢’¢ 


“p UOT}ONS P| B Sade}9 
xnap ud uoneUINSd,| ap soouanbgsuod sa] IMs suUOUdAII 
snoN ‘suotsuodoid say suep uoneLeA ke] onb Isure ‘sooumnso 
jUOS sayIp yuswsIdoid arpuodai ev suotsuadoid saq “s}UsIOTJIP 
sajsijiqeqoid sowsiuessul sop Ins sa9puoy} UONeUINSd,p 
sodejo xnop je} ud aystxod [I “(Z][) suep ‘anb suojoN 


(21) 
Y¥ Inoyeumso,| red y suodsejduias snon 
“UOISNIOUI,p splod say suost]yN snou no 
‘yt le! N 
(11) ——'d ¢— =d 
s ran! 


ee 
-B-1S9 9 ‘saautjsa orpuodal ev suorsuadoid sap uoyNUeYs9,p 
agigpuod suusfour eT dD og (O86 ssey) CIVHO 
UO]HRIJISSED OP SdIQIe So] yo (Z00Z BSesV) Wqoid no WSO] 
UOISSSIS9I IP sajapoul say ‘ajdurexe sed ‘yuos uoHeUIsSd,p 
d1U98 99 Ja[No[eo ap yUayoWUIEd Inb sapoyjaW sa] ‘19-sa]]90 
ap s[quiasua-snos un no sajqiuodsip samelixne so;qeuea 
Sa] soyno} asin mb ‘d ap imayeumsa un 'd jos 
‘guusAoul sipuodai e uotsuadoid ey] 39 sayjanprarpul sipuodal 
e suorsuodoid say nod asuoda1 ey] Ins sapuojy sinayewnsa 
sop Joslyn e ajsIsuod yuopIAg udAoW UL, “Y SIMoyeoIpul 
Sa] mod suonnjos sayne.p JaANO suoAap snou ‘yuanbssuosd 
Ieq ‘soquuoyjyueysa soytun sep asuods1 ap juaurayodwioo 
3] Ims onb sjuawioustasus1 sap suopassod ou snou 
‘sdepuos un suep ‘ano ug ‘sed yuos 9g] ou sala ‘anbryeid 
Ud “UOWIT[IIMeN “SONUUOS JUdTe}9 SaT[ONpPIAIpul sipuodal e 
suotsuadoid sa] onb gsoddns suoae snou ‘[°¢ uoNdeS BT VY 


asuodg.A vB] ANS S9PUOJ Y SANajedIpuy = 7°¢ 


‘d oxy 
suushoul opuodal e uorsuadoid sun mod (d — {)d/\ jew 
-Ixeul odAj-y1e99,] Jed sagsijewuou sipuodai e suotsuedoid 
sop odAj-ye99,] B yesa JUoWOANeUITXOIdde sa IOWIRID op 
A 9| ‘pueis so Av Is ‘onb suoAoa snou ‘(Q]) ap jueyeg 


LLL 


(dns DIN | Nf 
(OI) "COIS. Rett SERCO aN RAE = re = 


Ud SULIOJSUBI] OS (6) 
‘yuanbgsuod 3eg ‘asuodai-uou 39 asuodal : sau0s9\e0 xnop 
anb apassod au asuodai op inayeorpul., onbsind wy e [ess 
\So InayeuTUIOUAP J] ‘sed dQOU SUR “[ ISO d[RWIIXeU IMs]eA 
BS a Sadiadsa suorplodoid sa] 9dAR JUDUII}DVX9 JUSPIONUOD 
sagAtoasqo suorpodoid so] Is Q MoaeA ouN jUloye JOWIRID 
ap A XY] ‘aquaoel-snos sduasUTUOD ap 2] qv} LP] SURP SOUS] Op 
}9 SQUUOTOS ap dIQUIOU J] “WWoUIEANIEdSaI ‘JUOS Y 12 D NO 


J — {a ‘o}uru) ay 
(6) eae py) Tt 
G 
Jouiely ap 


A 9 389 (7007 Hseusy ‘a}dutoxe sed ‘noa) [] “Q] op[eAropUT | 
e _X anbysyeys ef WIOAUOS Inb UOTeIOOsse Pp aINsaut sul] 


(d -— 1)d 
‘(d pa IO Wi a 
(g) eo 
4 (d — pd 
d 4d) 4 = 
(d - "df i : 
(Grebpyeesaiay J I=¥ 
(do - DW 4 * o_O Ww & z 
(Oy Ny fait 


A Cele bol ana ti takes 


d 4 I=4 
Lusso reat a at 0 Ee 
AG Nome (Sic) x ; 


muayqo mod _X anbysiyeys ey arJoggi suoanod snoN 
“XY ouIg0U09 
Inb 99 Ud sIQIey sANeUSSoIdal ey e YWodde sed yRd9,] 
suoInsoul snou ‘Ip yuoWIanNY “Y op auepuedaput jso 
asuodai ve] puenb npusye jsa nb asuodai ap juauapIodui09 
3] 39 Jeol asuodal op jusuayoduiod 3] aqua soUR}sSIp 
eB] «Jamsaur» ap uye y e ,X oanbysyeys ey sonbijdde 
suoanod snoj ‘d jo “f saxif soyeursieu suonnquysip 
so] Inod onbyewoyjeur suas ne.nb oaielA s0uej\sIp ap 
UOTIOUOJ UN, p WIBe_.S dU [I ‘JUepuaday ‘sanagid jd sagArasqo 
suorodoad say aqua sourjsip ey ammsout _X anbysneys v7 


‘oy [— 
H 


@) (ROD ea en 
H 


Rae 8 
I 


1=4 
(d=atd tnd (9) .8 
H 


anb dSIJLI9A }Sd [I ‘SdT]NU JUOS IssepOe.QUI S9OUBLIVA 
sa] onb suosoddns snou anb ja ‘y sassejo sap a[quiesua.| 


6002 un! ‘a}gnbue,.p senbiuyse] 


INS « 9SSE[SEUI » JO « SISSE[IJUT »» SSOURLIRA SOP SUTUIOS 
eR] 1S9 srpuodsi e suotsuadoid sap soured ev] onbsing 

“4g = 'd ysa orpuodas 
e uotsuadoid ey ‘y= ‘Vv oaae 7 no} Mod ‘ou0g 


N 
(9) Ty sii § tee’ Inod ci op 


SIIP-B-S9.9 “Y SSeID 
R] 9p uoNefndod ap uonouoy ek] 19 sipuodal e uortsuadoid 
Re] “uoutsanosdsar “ff yo “4d jog “yy anbu0saq}e0 
g[qeueA oun Jed soruyap y sassejd say mod yuowoenbrun 
jusIgJIp ompuods1 e suotsuedoid saz aonb suosoddns 
‘uonenbape | 19 souepusdaput | 19}S9} mod dasi{yn yUSANosS 
yso Ib anuuod uaiq _X onbysiyeys ef & 91] JUSWIA}I0N9 je} US 
yso9_y onb suonuoul snon ‘sayuenbueul saguuop ap uoTyeaI0 
Op SUISTUBISW 9] 19 SJonbus ouN,p Wd} no} ayUS UONFIAI 
ounone 3}Stx9.U [I ‘[ = (d)y puend ‘uonelosse,p onbueur 
Np sIMsoul ouN jUe}9 SUIWIOD _Y IaJOpIsuod suOANOd snoN 


auudAoul 


aipuods1 e uoisusdoid el ap uonsuoj 


ud (S) Y ANIzVIIPUL| sp sjewIUM ANIeA = | IINSIY 
suusAow sipuodal e uorsuedoig 
01 8°0 90 r'0 Z'0 00 
— = —— oO 
iS 
wv 
ee) 
° 
San 
- Oo 
= 
o § 
an a 
co 
o 
= 
S 
“oIPULOW 4S9 


SHO} HOS say[onprlAtpul sipuodas e suotsuadoid sap uoneueA 
Re] onb syIIqQIssod e] ‘ofqrey osuodo1 op xne} un.p seo 
3] sueq ‘0 & Z/| op Jossed mod onurunp asuodgl ap xnr} 2] 
puenb ausuIsne sINdLIJJUI SUIOQ eI ‘WUSWOTexOpeIed “[ 3S9 
Q[PUNUIU MIVA PJ 39 B[QISsod JUSWIA\SazIUeUW SOU UOTFVULA 
sunone ‘T= Jd 39 Q=d Mog ‘CE 3p JewIuIW moe 
sun oapessod ji ‘7/[ = d mog (| amMsy ej Moa) osuoda1 
op xne} np puadap (¢) op syeumutu Maeda ev] onb suoj}ON 


(s) HO) Siscn [eee (GQ) or 

Jed tuyap ‘y sed suoudisop snou onb “y mo}eoIput | 
suosodoid snoN ‘opHo} ayANeUssoida1 ey e yodde sed 
[EUMTXPUT 1199 | JUL}D () INVA LT 39 BOF DIANLUSSoIdoI oun 
yuejuasaidal | moped ey ‘[] ‘O] ayeArour.] suep sosuduros 
smgjeA say suusId Y mMoajyeoipul., onb suojnoA snon 


angojeye9 nev X-[(00-ZI .N ‘epeueg onbnsneis 


Z 
() : min(dimnp)d Sisu(d)s 
aonb JaNUOU ap a 1OIJIp sed jsa.u T] 
I=! 
="N 
(¢) ‘(d - 'd) i ret Aids 
N 


asuodal ap sayiqeqoid sap adAj-ye99 | & 
gyjouuonodoud ysa amsaut ayja0 ‘°d 39 d anua souR)sIp oun 
e sanbiddy ‘ajduns a0ur}sip ap UONdUO}J oun jsa sUUSIpIjON 
Q0URISIP VT “sJOI unone onof ou ayeqo[s asuodal ey] ap 
Iayney e] aonb suojoN “(2d ‘d) p yuR[NoTed Us aANEyUasosdaI 
yuswspioy osuods1 oun e yodder sed yreo9] Jomsou 
Be yemgns [[ ‘o] suep p souejsip sp uoouoy anol 


‘uoyejndod 
eR] op ouushow sipuodsi e uorsuadoid ev] ap amnsuos 
Iaj00A 3] d x [ = °d pOS ja ‘T saea sp AV UOTSUaUNIpP 
Op Ind}90A 9] (T ‘*** ‘T ‘[) = I os ‘ompuoda e suotsuadoid 
ap inajoaa un (4d ‘:*: “Sd ‘'d) = d jog ‘apo; Jsa UONPLRA 


e] onb oipulou! juRNe,p jsd YO} suas ne s}IANeJUOSoIdaI 
eR] ‘ srpuods e suotsuodoid soy suep uoneueA op 9Nuenb 
PR] JoMsSOW B JUSWa]duIIs sUOYdIDYS snou 39 sO} UONTUTJEp 
PR] 19}Sd} SWIgWI SUOANOd snouU ‘sed 99 sURp 4UdUd}soyTUR/] 
‘sonuUOS jUOS say[onprlAIpul ospuods1 e suorsuadoid soy 
no onbysyjodAy uonenis ve] ploge,p suolapisuod snoN 


uoneindod ap xy sano}edIpuy =[¢ 


"saoulysa apuodal e suorsuadoid sa] 
Ins }9 UOT[ULYSE UN INS Y IMaedIpul SUIQUI 3] SUOSeq SnOU 
‘sing ‘uoyeindod op y Imoyeorpul un suousid snou ‘Ia9u0U 
-W0d Mog ‘anbyeid ua sanuuoout juos Inb sayjanprarpul 
asuodai op sayiqeqoid say] ms juatndde.s suoniuyep xnop 
SY] ‘oANeUaSoIdo1 JUSWUATQIe} 39 SANRUSSOIdaI JUSUISLOJ 
asuodgi oun IuIjap SUA snou ‘aJUapgogid UONeS ek] V 


Y Sanajyeopuy *¢ 


‘VY op aouRssteuUod FR] Ins JUePUOJ snou US 
syuepuodsl-uoU S92] Ja sJUepUOodaI Sa] BUD UONOUTSIP ke] SIIeJ 
sed suoanod au snou YWvOW eseujoddéy,| uojas reo Sy ke 
yoddes red yD 2dAy op soyuenbueul sssuuop op uoTReI0 
ap oulstuvsou un e puodsal0d s/qIey UOTIUTJap eT 


‘Q1AO0B9JDI 
ajjao supp sajlun sa] sajnoj ans ayivf jsa auwos v1 Ja 
y asspja bj Supp ¥ ajiun,] ap a4puodeas v uorsuadosd Hy 18a 
“4d ‘y a14o8aj02 Db ap uolojndod v1 ap api) vj sa “NE no 


=X 47 
(Z) Cr re hee 4b) Inod qr %d iG ore 
‘N 


sajenbus xne asuodai e| ap syIANeyUaseidai e] ap SINayeoIpU] : WaYs|yJog Je UBqgOD ‘Us}NOYyoS OLL 


anZoyeiwo nev X-[00-ZI .N ‘epeued onbnsneis 


J1OS ‘aJUDJSUOD JSA Sa140BaJDI 
say ans auuatou aspuodas v uoisuadoid v1 18 Sal40sajoo 
H{ juvpassod x anbtsosajno ajqniapa aun p fiyojuasasdad 
jsa sasuoda4 ap ajquiasua-snos up : (ajqivf) uoluag 


‘anbyyeid ua 99}S9} a9 jnod Inb s[qrey UONIUIZEp 
aun Issne suosInyjsuod snou ‘uanbgsuod seg ‘onbrun 
gutun oun.p osuods1 ev] ap uonneds: sunone suopessod 
au snoN ‘onbneid ua 9syIpiyea es 1a}so} op osjqissodum 
\SO [I ‘ojURsInpas Wos uoNTUIZap ev] onb usrg ‘saqissod 
ajgnbua,p suonsenb saz saynoy mod (YQWOW) presey 
ne yUowslaue yuonbueyy add} np esos [I ‘oyoy UONTUTZSp 
eR] yejsyes soyuenbueur saquuop op ouWIstUROSU UN IS 


‘SOJ1UN SAAJND Sa] Sajno] ap 
asuodad dD] ap ajuppuadapul Jsa ajiun aun, p asuoda. vj] 18 Ja 


(1) IA O=[P="s| T= “dg =" 


uoyojndod nj ap sajiun saj sajnoj anod sawau saj juos 'd 
adpuodas p suoisuadosd saj 18 uopiyuvysa J ap fijpjuasasdad 
jsa Ssasuodas ap ajquiasua-snos uf : (ajsof) uoyiuifag 


“QUO} UOTTUIJOp duUN sUOUUOP snou “Iad.UdUTWOD INOg 
“O[9pour UN Ins spuoj} 
9}X9]U09 UN suep sjiInpey a9 JUaAned s}daou0d say sno} 
‘yoyo .p nod un jueuusXour ‘siojaynoy ‘aytun 93399 ap aqei 
-edasut “uanbgsuos sed ‘ysa jo ‘aysod ua apres gy1un.] aonb 
dasieiq sIeuUOU ap adeId oun ap nad uo] Is ‘ajqeyyMUspl 
Jo dgjonbya aun oun,p onbysuajovleo oun ysa osuodo1 
ap ayqiqeqoid sup ‘amoyegye orajovreo un yuo osuodal 
op SINdeOIPUI Sa] 39 UOT]HUeYDS,| sjnes aonb oiyrusis mb 
99 ‘ajapour Jed aaqsisse ayooidde oun suoatns snon °9}09IIp 
sed 3so.u oipuodai e uorsuadoid oun.p uonroidiaqut, J 
“A9UUOT]HULYIE 4Sd a9 puenb spuodal 
1 gqun,| onb oyyiqeqord ey ‘d yog ‘sayjenptarpur erpuodes 
B suotsuadoid say suep dpIsel SUOT}IUTJap SOU ap 9Id eT 
2 QIU] 9p dIpIO JatWIOId op UOISNOUT, p ay]Iqeqold P|] 
ousIsop ‘u ‘UIJUq “U 4Sa UOTTMURYS,p a]Te) eT “GQ ISA IMaTBA 
es ‘UOUIg “[ = 4% sioye ‘puodal ayja.nb 39 sguUOT;HURYDO 
so 2 QIUN | IS 7 gUN,] nod [-gQ asuods1 ap moajeorput | 
4 Jed suouSisop snoN ‘yuowonne Q 39 ‘aauUOTTNURYDE 
}SO [JO IS [ Moyea ey puosd 7 gWTuN,] onb amp-e-}sa.9 ‘[-¢ 
uoT[HUeYSS Pp InayeoIput.| ‘s red suousIsap snoy ‘uoNe|ndod 
eB] nod gyun,p sayenbng sot N ‘** ‘E ‘7 ‘T=! UOg 


juejuesoides 
sasuodo1 ap dJquIasud-sNnos UN.p UOHIUTIG = 7°7 


“satqissod 
SOZIPM9 SoTqeUBA Sd] S3]No} dUIZDU0D INnb 39 Ud YWYOW 
odA} op ousTURSOW UN SUTOD No ‘s}uBpuods sap UONDETaS 
B] SULP SdATJOB[9S SOdIOJ Op doUaSgR,]| JURTFIUSIS sUTUIOD 
Meidigyul ang ynod gou0Ud IOIWIOp a] ‘soNe sap soun 


601 


So] saytun say JuansuNsip nb sanbysug9e1ed say 39 osuodg1 
eB] anus sfqissod onb s]qQIey Issne uOoNeoI oun juUR}UOSOId 
dIIp-b-1S9,9 ‘« aJgnbua,| Inod gstyyn uoT[HueYys9,| op ojdums 
anoyegye uoyHueYyos» un.p aiqissod onb osysoid Issne 
JOS syuepuodal sap uOT}d9]9S eT ONb sUO[NOA SNON “o9UTUTeX 
ajgnbua anbeyo snod sgXojdusa ag yuaanad nb ja ajygnbua_| 
ap aiped np sioyap ud sdArlasqo juos Inb sojqeuea op 
stuljopaid sajquiasua p uaAour ne asuodail e] ap uonIsodur0s 
e] Josnf ap suosodoid snoy ‘osuodai e] op oy1Tenb ey] Jonyeag 
Inod sagsijyn sanusa,p sdguuop sa] JoyUasgidai sed yudAnod 
ou sl ‘ana ap jyutod anou op ‘uanbgsuos sed ‘9 939)S89} 
Tu d9Insow ane sed ynod ou souanTUI 9199 seu ‘o}gnbua,| 
e juodionred saseusu say aonb syI]Iqeqoid ev] jUD0UNNy 
“UI sgIpmyg sjo{ns soy JuoWoNe[D “UoTTHURYD?,| IP 999 
e (syjuepuods ap adnoiz) ssuodai ev] ap UoNIsodwios ey] Jared 
-WI09 B JUDUDWIe SNOU SUOTIPUOS sad “UONeUINSd,] Ins onb 
}O]nN|d sasuUOp sap 3}99]]09 e] op ayyenb ey] IMs JanuUaOUOD 
snou suolJgjoid snoN ‘dsiyyn InajewmMsd,, ap suepusdop 
yue}d SWS TU yp yUoUIOIdoId ajanbua.p jofns np suep 
-uodap jue}9 sWIUIOD dANRIUASoIdaI asuodaI sun ITUTZOp op 
JURSSOINIUT Sed 4Sd_,U [I ‘S99]Nd9Xd9 JUOS SaT[> No spotiod ey red 
no o1pme jofns a] ed yusIayJIp nb saygnbus sop e asuodoz 
eR] Jaredurod 39 Jay]IaAIMs suOIISop snou onb suuop jue} 
‘qUdINIP UONPoIFHes oun 19}I1ssa99u yeumod 
wot anne uA ‘uoNetndod ey op ssarnonred uoneoynens 
oun mod jyuon way un mod yy oulstuesoW un.p 
asoujoddy, | ‘anno uq YVAN No yy odAj ap qos saynne_p 
onb spur) QVOW dA op ane juaAnod suaqI sureya9 
‘smreuuonsonb suguI uN sued ‘JaNUaSsd jsd aJonbua,p yafns 
np ynole,J ‘syjuepuodsi-uou sap syualayjIp juos syuepuode 
say ‘suonetndod-snos op utas ne swgut ‘anb onbryjdumt yWINN 
9 ‘syuepuodgi-uou say aonb sawigur say auuaAOU Ud jUOS 
sjuepuodal saq ‘sanuuod suoneindod-snos op utes ne ‘onb 
dIJIUSIS ZYW ‘syuepuodsi-uou say onb sourgu say ‘ouudAOUL 
ud Juos sjuepuodsr say onb oayiusis y~yOW ‘aenbus.p 
jyofns un e yey ev Inb 99 ud yUdWIDIGI] sutour no snyd d}91d19}uT 
‘UONST[9POUL B] INS s9puoy onbysHe}s aU09y) ek] DUISUIO mod 
JUO SOUWSTURIOU $90 Ye} UT YVAN dA} op jso oustuesour 
g] ‘sqyuenbueuwl sosuUOp sap yuoWayesa puodep s]Iqeqoid 
BT IS “MVOW asomjoddy,, anb sqrey snjd asgyjodAy oun 
ouop sod Inb 90 SUdUIaTN|aS SadAJOSqO SaguUOP sap puadap 
asuoda1 op ouypiqeqoid ey Is yy odd} op jso ouIsTUROSUI 
YX] ‘1g UI,p aJonbus,p jafns np sed pusdap ou asuode1 op 
aTIgeqoid ey puenb yYOW dA} ap iso sajyuenbuewl ssguuop 
op uonesId op suistuespW ur) “(ZOOZ) MIQny 19 ANTI 
TOA { aINAVIDT] Vl SUP SSUUONUSW JUDANOS jyUOS INb (presey 
ne sed yuonbuewl oN no wopupy-jn-suissyy-JON mod) 
UVWN 1° (presey ne juonbuepy no wopuny-jo-suissipy 
mod) yyW ‘(presey ne yuoWsraNUS yUonbuey no wopuny 
-Jo-Ajajajduo)-suissyy mod) YWOW seusisop soyuenbuew 
so9uUuOp sop uUOnedId op souwstuedoul xne jUoWO}ION9 
Qo issne jso oatyeyuosoidor osuods1 op ydaouod a] 


600Z uinf ‘ayanbua,p senbiuyos, 


“xIOYyo 
39 suosIe} snou 1onbinod suonbrydxa snou ‘eya0 yURAR SIRI 
‘« SOATJOO[IS S9dJOJ SP dodUasqe »,] B puodsauo0s Inb syANne} 
-uasoidal ey] op uonluljap oun suosodoid snou ‘ayuearns 
uoNdas ve] Y ‘OUUOSTIeI adeUUO][HUeYSS, | 19 eIONb Ied a8eu 
-uoy]nueYo9 | ‘qstiqeqoid oseuuo][nueyss | & yUaIodde! as 
« xneapl no sonbidAj » seo ya « UoNeNdod ev] ap omjeruru » 
‘« SOATIDIJIS SIO} IP DoUdSqe »» SIDUIWIOU JUO IIT[AISOJW 
}0 Jeysnry onb sanbysyejs suonejsidiaqui say ‘sanbnsneys 
S3}X9} SO] sUep saquasoidtuUO jUOS suUONE}DIdI9j}UI S39 
Op WQUIOU UTeYIIN UA) “sa}oUT|SIp suONRIaIdIO}UT Jnou dAIIAI 
19 AINVINT] ey suep « JHeUasoIdaI»» Jou Np uOHeEsTHN,| 
op ipuojoidde ameyusaut un e uapadoid (9 19 q ‘e6/6]) 
JgJaAIsOopPY 19 jeysnry ‘anbysyeys simeisyy ey suep uoy 
-RISOY SDA YSI[NN JUSANOS 38d « JIJeJWASaIdaI »» SUID} IT] 

4 Jueuosoidas Jed snou-suopugjus nb 
‘g7e1ouad SINTULU OUN Pp SIey ‘SoeI]Ixne sajqeuea xnop xne 
yen e nb 99 ua dATEJUSSoIdoI SUTOUT asUOdaI oUN eB YNOGe & 
Tu990e YOId,| onb yWowaNe[s JoNUOW aTquas a]duIaxa J 


% O01 1G POS UNO LY esuodel op xne | 

% OSI MS CVG EG Cl jueu uou yuseg 

WZ WHOL WSO safe1s0s suonedso]|e sop jUeAdey 
SIOUl 7 SIOW | 


uoTgueysy seaidy saidy aquuiva 


SIOW! XNIP IP 9}99][09 ap 939[,dwI109 apotsad e]| Anod 32 9}99][09 ap 
slow 19twiaid aj nod sJQqd agnbua,| & sasuodai sap sauushofyy 
[ neo[qey 


“stow Jorwuaid a] soide nb yueyodun snd sterq un 
juajuasaid ajanbua,| ap saauuop sa] IMs sagpuoy suOTFeUIT}SO 
So] ‘sIoul oWdIxnop os] saidenb onuow | neo[qe) np 
uowexd | uepuaday ‘asuodai ap sameyusuistddns %% ¢‘Z] 
QUUOp & MOIAIOQUI.P SIOW dWATxNop I] ‘NUOG ‘OVd] Jed 
9109]]09 Sp Spoul uN kB sa{dafFV 919 NeIANOU dp JO SI ‘UOUTS 
‘otjqnd j1e}9 O1guINU 3] WUOP TY ed anbruoydsya} dd1AJ9s un 
& sue JUSIE}9 S]I,S (INdyeuIpsO ed daysisse onbruoyds|s} 
MOIAINU) OVI] Jed aoajoo ap spow un ke sa~aye 99 
quo syuepuodal-uou say ‘stow Jorwaid a] soidy “(ana}euIpso 
Jed ogystsse aorjd ms MdIAIaUT) OVdI Jed samsayo 
319 & 9109]]O9 eT ‘stoul Jatwld J] ‘8661 2P STOd Aenbua | 
Op Sed 9] SUR] “% L*6S SP 31eI9 [I “SIO XNOpP 9p MIIAIOJUT_P 
a19[du109 apouiad ey] soide nb spur} “4% 7p op wWe19 ssuoda1 
op xne} 3] ‘slow un saidy “[ neajqe} ne sosuuop jUuoU! 
-g[e39 jUOS UOTTWURYD9,] suep suoTIodold sa] “syuepuodal 
-uou so] mod simajea sinay yueuTddns us ayenbus oun ke 
sasuodal Sap SUMS JUIWIT[ITOIILIE Sd9TVI] JOS 39 So}sIdoI 
ap soyenxd jUOS saguUOp soy ‘SoTqeLieA xnop sot Mog 
‘seg-skeg xne nb sinayyie ou jie}a JUored un sulow ne juOp 
uomodoid ev] ap jd sayeId0s sUONLIOTIe,.P SULIOJ 9uN WeADIOI 
Inb asiepuejis9u uoneindod re] ap uonsodoid ey] op ‘slow 
xnap saide ja stow un sade SJOd ajgnbus,| e sosuodg1 
Sd] INS saapuoy ‘suONeUINSS So] JUITUOS [ neque) I] 


angojejye9 n& X-100-Z1 ,.N ‘epeued onbysyes 


| (siesuey 
Ud SOSLUDSUI SdP IIA IP SUONIPUOD sa] Ins ddI89}UT aJgnbuyq 
no ayonpsfaaT Yyaozapuc—) JuauDuuiag op suwWiAUOIOR) 
S7Od ostepuepisau ajonbua | sp om) ajdwioxa un.p uadoul 
ne yutod 99 yuoWaTIORy JaNsnyI suoanod snon ‘sajgnbus 
sop ayyenb ey sonjeag Inod asuodai ap xne} say Ins syoyyo 
SOU SNO} JoNUDNUOD sed dU Op IWUSAe SNoOU aINjeIDYI] eT 


é JoeUIsgAd91 JIIUSIS ang) ‘7 


‘UONIUIJap oun yURUUOP IN] Ud xnoInosU 
juswonbyewusyjeU ayAneyuasoida1 ap ydaou09 2] suopual 
snou ‘oyInsuq ‘uoT[WUeYDS,| op Jueyuaseido1 sa ajanbus 
oun e& sjuepuodsal op ajquiasua un,nb suosip snou puenb 
suopusjua snou onb 39 suonbrydxa snou ‘nary Jamwaid uq 


dAQEjUISII1d941 asuod9.1 ap 3dadu0d aT “7 


‘UOISSNOSIp oun 
suojuasaid snou ‘9g uodes Pe] & “UJU “UTeI9} 9] IMs sapmo 
xne Y INgVoIpul a9 op uoNeordde | yap ¢ UONdIs eB] “Y 
INIVIIpUl | sp sonbysuasered xne sg1OVSUOd }Sd p UONTIIS 
ey] “IMaevoIput anou mod onbynewsyyeu UonLjoU ey] suOssTU 
-IJap snou ‘¢ uoONdEs eB ke ‘sIng “oANRUasoIda1 asuoda1 ap 
ydaou0d np Janosip Jed suosuswmu09 snou ‘7 UOTdES eT VY 
‘sonbuiduwia sopma 
yo sajdurexa sanne,p mod (/90Z) uanoyssg ja uaqqod 
e.nb sure (7097) Uaqqod 19 UayNoYS ke M9}99] 3] SUOAOA 
-UdI SNON ‘Sashyeue Sop SUOISN]OUOD xne Y MdVoIpur | 
ap sINa]eA Sa] suoIeduI0d snoN “Y INayeoIpuT,| op onburduwia 
uoneplyeA oun Imod uaiq yUdUUdIAUOD sala ‘Uonbasuos 
Jeg ‘syoddei ap uononpoid ev] ap 39 saouUOp sop dgT]Ie19p 
askyeUe dUN.p SIAINS 9}9 & SaPMea sad op suNOeYD ‘asuodas 
-UOU ap Sed Sap IAINS ap SaIsa}eNS SSIOAIP 1d soguUOp 
SOP 9199][0D Ip SapoWl SIDAIP JUaTeyOduIOS sa[q “soouUOp 
SOP 919d]]09 ap saisajeNs sayussayJIp Joredwuio0d op synoolgo 
Inod yuareae saq[q “9O0T UP 19 SOOT US seg-skeg sonbnsneis 
B SOUSUI SOpmMe Xnap & Y IMayeorput.] suonbrjdde snoyy 
"sajoarip sonbrysug}ovses 
sap sed juos ou uONesITeULOU PR] 39 UOTeJOIdI9}UT | onb sored 
ynoyns ‘99 1q ‘aInsaw ouN.p sonbyeusyjew sajouUdold so] 
yueke Ud 3NO} dST]eULIOU 39 aINsow ‘a}9IdIa}UT 3439 ynod Inb 
Y IN9}eo1pul UN suOTNOA snou 4Ip JUsWAyNY “omMsow ouN.p 
sonbysuajovied soy IIOAR JUOAIOP 19-xXnod “TMUSAR.] SUep 
sajonbua sap aytjenb ey ap uostereduwo9 ap 39 S0URT[IOAINS op 
S[MNO SUIWIOD Y SINdyeOIpUI soy JasI[N WoAnod op uy 
"S9dAIOSGO SddUUOP Sap UOTISOdWIOD ke] d9a]ye & asuodaI 
-uou ey onb aanaid ounone 94s1xo,uU [I,nb oIfusIs e[90 Io 
‘a]qQIe} ISO UONRIDOSse,| ONb oIMd[[IoOW jURINL.p So UONEeNIIS 
eT] ‘UoNeIDOsse.p onbuewl np sMsou suN SWIWOS Y IM9}edIp 
-UI,] SUOJOPISUOD snoU ‘je} U_ ‘SoTelIxne so[qeLeA so] 39 


sojenbua xne asuoda e| ap ayiANe}Uasaidai e] ap Sunayedipu| : WaYsjUujeg Je UBqgoD ‘Ua}NOYoS SOL 


‘[U'SqOMUysq : [BLUNOD ‘spuLpyiayIeN ou, 


‘Z1InQlooA WE OLEZ ‘000% XO Od ‘Artjend pue AZojopoyjap] Jo juswedag ‘spuejyayIeN sonsneis ‘WoysTyIeg ay4Jef 39 U9qqoD sluuRy ‘uajnoyos Aueg *| 


asuodal Pe] aUD UOTRIDOSse | INod I9WIVID Ip A MSO RT & 
auoredde jso ssodoid y majeorput.| onb suomuoU! snoN 
“squuenbueul sdguuop sop UOT}edI9 
ap ouistuesour np jofns ne sonbi089}e suOIsn[IUOD sap 
Jom op sed yuawraydurts yno} yuoWouLINd ou sajgnbus saynod 
Sa] { aINsaU 93N0} Inod aystx9 souepuddap 2)99 ‘Juepusday 
“SUOIDIUOUW J] SNOU SUTWUOD “Y INDBIIPUT,| Ip UONeNyTeAd | 
suep jueyodut sjo1 un suop onof ajgnbua oun.p uo 
-ueYO9,| OP a]Ie] eT ‘sIeIq uN yUOWAT[OMUDAN 9 UOTSIDOId 
aun apassod j1,nb “uanbasuos sed 49 oojeaye s[qeLeA 
aun 3sa yp yuauoidoid y majeorput., onb onbrydut sayyIq 
-eqoid sao ap uoneUIsa, J ‘sasusa osuodai ap sai{iqeqoid 
sop ins omdde,.s suosodoid snou onb y moaeorpul,T 
‘asuodal P] 19 UOTTHURYSS ,| ou 
SOURTqUIOSSIP P] Op d[OWUOISUDUNIPH|NUT sANDodsiod oun 
auUOp dINSdU PT ‘ANNO U_ ‘saddUUOP Sap 9199]]09 ap s}UdIIJ 
-JIp Sapour jo sarda}eNs sap Jaredwios mod onb sure ‘sdui3} 
np sinoo ne ajanbusa oun,p sjey[nsai say No sayja aqua saygnb 
-Ud SOSIZAIP Jaredwiod mod sjyno sUIWIOS JasITWN say op 
youuod Jalnosed answered un,p sMoayedIpur Say JOIDOSsIq] 
‘gsuodal ve] ap ayeqols uontsodwiod ev] Ams jajJ9,] & SUOT[AI 
3] Snou stew “Iaynomsed uonejndod op anouwrered un.p uon 
-PUINSI,| AP dTANeJUISOIdoI ap ydaouod a] SUOTOSI SNON 
‘uonendod ap sauuadour sa] inod 7 snbnsne}s 
P] & I[QUIDSSOI SLIBATUN SILO 9+ “dTATeJUASId9I Op dUID}UT 
argu un ayodde j1.nb 99 WuyZAp (L007) [YOM “Sasty OA, 
-INWIBID op onbysyeis e] Ins s9seq UONNaLASIp op ajduraxo 
DINSIUI SUN jSd DOIPUL Jad “SONUTJUOD saMo}eg]e sayqeLiea 
sop Inod gueAtun jp dIANeUSsoIdoI op sdIpUT UN UTJOp 
(900Z) OUTLIOg “sodTed JUOS INb someTIXne soyqeLeA soy nod 
synejuosoidar yuos (EQOT UeUNDIAA 10 WOSsUdMS ‘TepUreS 


‘gsuod9J-UOU Be] Op UOT}DaION { asuOd9I-UOU P| 9p UOTJONpal { asuodaJ-UOU ‘ d}I/eENd 


‘gjduioxa sed) a8eyeo red smayeunso so] “yofe] op uonIUTjop 
eR] JURAINS ‘uoNnEINdod op anowiesed ne [edd sd INd}eWN}Sd | 
‘un op ayyiqegoid oun dane ‘Is dANRJUOSaIdaI Jso soyUeNbueU 
SddUUOp op UONeaIO op oUISTURDDW UN 39 Md}eWIsa uN Jed 
dou} amted vy { UoNendod op sanowresed ap uoneuUNsa,| & 
« JNRUASoIdal » SUID} J] S191 (1 86T) AoleH “suANeussoidas 
RP] JUoWaIOI[dxa jUassIUIJop sInayne suTeyad ‘onbysyejs 
dun, p SIvIq sURS UONRUIT}Sa oUN ITUd}QO,p JURYOULIOd soyeU 
-IUTU SUOTIPUOD Sd] dTIP-k-]S9,9 ‘a[qeIOUS! dsuOddJ-UOU op 
jdaou0s 9] WNpoNut e (9/61) Ulqny ‘onbnsne}s uou 3a onby 
-Se\S sINjeIDI] ev] op stpuojoidde apnjo oun mod (9 39 q 
‘26L6[) JI[OISOW 19 [eYSMY Jaynsuog “« asaneiussaidal »» 
ap }daou0d np suonejaidiajUI sasnoIQUIOU dp IJJO s9SIT[RIO 
-ods ommelont] ey “« aatjeyuasaidoi asuodai»» agjadde ane 
ynod ues ayag “ued anne .p ‘opnja| e uoNetndod ey] no 
uo]]HUeYD9,| 39 “ued auN,p ‘ajJonbua aun e asuodal ey aud 
uULUNS eV] op ‘(azANeUISoIdd1 Mod « yY») Y IMayeoIpul 
suojodde snou onb ‘majeorput un suosodoid snon 
‘qyonbus oun ,p 
ayyenb ey] ap smayeorput nb jue} ud snsiquie suToUr JUdTeIOS 
mb asuodsi-uou ev] op SsMa}VoIpUl sanne.p I]qeia o109Ud 
sed jU0,U ouTeWOp np sajsiTeloods say uepusdadg (P00Z) 
uaynoyss onb isure ‘(/00Z) IPJaAsooT 39 siqy ‘YsaMII0H 
(ZOOZ) URWTOPY 19 SPPEWA “(O0OTZ) ‘1109 19 JaI}993 “(900Z) 


BAMYNAIG 9 SAACIQH “(90OTZ) SPACID “(HO0T) OAdId 30 
IISSaId ‘S2AOID ‘(QOOZ) JosUIS 39 Jossaig “uIND ‘ajduraxa 


Jed ‘moa ‘ asuoddi-uou op siviq np sinayeoipul sa1oo1psul 
ap ‘sjnas xno e ‘uos asuodsi ap xne} say aonb ajgnbua.p 
ISO[OPOUIOU PT INS sINWVIOI] e] SUBP }LIO9P UdIq 3S9 |] 


uoHoNpo.yUy *T 


: S9[9 SIO 


‘sonbyeid sojdwioxa xnap e y Inayeorpul | suonbijdde sno ‘saguuop sap 9}99][09 
OP SOPOU! SIOAIP NO UTeLIO} 9] Ins aJgnbua.p saidgyes sosi9AIp Inod no ‘sdwi9} np uoTJOUOCJ Ua sajgnbus xne asuodal Pe] 
op asdyeur | Joyploey ynod |] ‘osuodgJ-uou ap sieiq 3] Jonjead e JuaWIa|ediouud suNsap jsa Ja asuod91 ap xne} xne juaUIa|duI09 
9p Atos ynod “y InoJeoIpul suojsdde snou onb ‘ayyenb e| ap Anayeorpul 39D ‘aJanbus 390 ap uOT]NURYOY,| 39 aJanbuse sun 
B 9SUOdSI PR] SUD ILLIIWIS e| JON[AI,P INA Ud INo}eOIpUI [aANOU UN ja s}dadUOd ap dIqUIOU UIeLIZO UN sUOSINpONUI sNON 


(pOOT UaINOYIS £ NOOZ Jassed 19 SPADIDN WNYyoy 


‘IST “191099 £ 9OOT BPADYDIADY 39 SOAOIDH ‘o[dwoxe sed) sinjeigyI] ve] suep sojdwiaxa xnoiquiou op piesa jo9 eB dANON 
UO ‘a]qIe} Sa 9suodgJ-uOU P| B NP sieiq a] onb sed onbijduit.u 9aaj9 ssuods1 op xne} un ‘[e19uas Uq ‘asuodgJ-uOU ap sIeIq 
Np InajJeoIpul uog un sed jso,u ssuod91 op xne} 2] ‘[Nas In] eB “Uepuadag ‘a[quidao0e anf neaatu un e dsuodai ey] JUa}UTeUL 
ap no asuod9J-uoU ke] dJINP91 9p ONA Ud SoINsoU SdSIDAIP DUOP JUUUaId S|] ‘osuOdaJ-UOU ap sIeIq NP ja}J9,] 9WIIOUOD INb 
99 Ud JasI[yN & syyenb ek] ap INo}eoIpUr,| 3UeI9 OUIWIOD dsuOdd1 op XNe} I] JUDJOpIsUOD sanbyse}s sowstueZIO xndIquiOU 9q 


UNS IY 


, WaYya[yIIg BJef 39 U9QqoD otuULY ‘UdJNOYIS ArIEg 


sajgnbuo xnv ssuoda. ev] op 9}1AI}8}UIS91d91 L] IP SANs} vdIPUT 


Z0L 


anBojeyeo ne X-100-Z1 .N ‘epeued anbiysneys 
LZL-ZOL “dd ‘| N ‘SE “1OA 
6002 un! ‘ayenbua,p sanbiuyoa 


EL Leb Epl “VW AU9S “alz0g poousunig jodoy 
ay] fo Joudnor ‘Kaaing Aj W194 POA, dy} 10} sJous Surjdures pue 
sudisap adureg (0861) ‘D ‘ysteyRayorINW.C 19 ‘D ‘Noog “A “eUUa A 


'80S-66r ‘Z8 ‘suljdures 98e}s-om} 
peyeys Ul suoloUuNy soURLIBA poZI[eIoUsyH (1861) WY ‘welled 


'S6-b8 ‘Lb ‘Ap4ajanng uoluidC 
aygnd ‘sK2amns suoydsya} ul sjdajjo JOMIIAIOUT “(€86]) ‘D “axon 


“OUT ‘SUOS 2p AdTIAA UYOS : YIOX MON “Sdaaung xajdiuoyg 
Jo sistjouy (6861) (SPA) “AW'L “WtUS 19 - OH “fD “Jeuurys 


"66°68 ‘8h “A SHES “a120g9 jpousimIg [oAOY ay) 
fo joudnor ‘Burdures a3e}s-0M) JO JDaJJo USISAq (9861) f'D “OUUTyS 


‘anbisyoj¢ ausiuD3sicg) un p Saauuog sap ajijonO VT ‘I00Z 
wnisodudg : jianday ‘sameuiq sasuodai sap Ins INaMIIAIOUL,| 
ap sjaya sap uoneunsy (1007) ‘dd ‘staeq 9 “fy ‘nods 


“OI 68E ‘17 ‘sousums jo191f[C fo ouanor *s\da}J2 yuLod-But|dures 
pue JomotAlgqur Suyeredag (S007) “4 “Jomery yo “Y ‘Touyss 


09-9p ‘ZI ‘sousuwig fo spouuy 
‘eyep AdAins WIOY poyeurss suotodoid YIM sajqe} AouasutyU09 


ABM-1[NUI IOJ $189} porenbs-1y9 UO (P3861) ‘['V ‘NOIS 19 “WN'f ‘ory 


“€@t-07P ‘SE ‘A80j0190¢ fo joudnor uvdidaup °d\0U [edISO]OpoyoUl 
V :MOIAIOIUI oy} Ul seIq snolsewoD (6761) “WS ‘ord 


‘Weplo|surY “noe QZ-/] ‘Sofeloos 
SOOUIIDS SOP dIZO]Opoyjou ke] INS dD[eUOTeUID}UI DOUDIOJUOD 
QUIZIXIS BP] eB aUassId spONTY “‘AdAINg [eId0g uesdoinq 9y) Ul 
OURLIVA PdJLIII-JOMIIAIOIU] “(PO0Z) ‘DH ‘IP[Pasoo7T yo “py ‘suoddiyryg 


LES9 
“IQI ‘WV Salas ‘Aja120g jooysyojg jodoy ay) fo jousnor “uotsi9e1d 
Adains Uo sjaJjo usIsop sjdures puke sjddJJo IJOMOIAIOJUI JO 
yoedunt oanejas oY] (8661) ‘d ‘UfouedureD 30 “> “ysrepeoyosnj.O 


89 
“1 ‘(8S61) ptyyung suep 911p9-21 “8LE-SZE “601 “VW 2H9S “dal20g 
jooysiwisg jodoy ays fo jouanor ‘aymyysuy] [eONSHeIG URIPUT 9) Ul 
Surjduies jeoysyejs ul syusuIUadxa yUs99y (OP) ‘O'd ‘SIqoueleyepy 


JPd'L0-po0zgPd/sdedyiomysqnd 
P{Noe'xasso'Jasi “MMM //:dyYy ne sjqruodsiq ‘xossq Jo AjIsIOAtuy) 
:INSIYIIOD *LO0-pO0T Joided ‘youvasay 1uo0Uu0rq pup jp190¢9 40f 
ainqysuy ayy fo sdadvg Sulysofy “BULID\SNID 0} ANP $}daJJo USISAp JO 
UOYLUITSS JY} Ul yg 0} suoNeWIxoIddy ‘(pO0Z) *S “J91qBD 19 “'g ‘UUAT 


“OUT 
‘suog 2 AaTiA WYOL : YOK MAN “Surjdung Aaaing (S961) “T “UST 


SL 
-Z6 ‘LS “UOIIDIIOSSP [DINSIIDIS UuDIIUaUp ay] fO jDUANOL *S2|QRURA 
[eUIpTyMe JOY VoURLIVA JOMoIAIaUL JO saIpmg (7961) “1 “USI 


angojeye9 ne X-[00-Z1 .N ‘epeueg anbysneis 


Jpd'g iadeyogpdshoamsyyypsun/s10-un'syeysury/:dyy ne jqruodsiq] 
‘IA anideyo ‘sazgunoy uoyisup1y pup suidojasaagq ul sdaaing 
ajdupg pjoyasnoy] : Sueq “usisap a{dures ut asn Jo} s}oayja UsIsap 
Jo sjusuoduios Suyeumsy (S007) UL 8T ? W WU “DH ‘uoyey 


‘PPre-6TE “AuUONG » AjyjoNE “siskyeue-eJOWI 0} Buljapour 
jeas[utnur suiAjddy “sXaains s9dej-0}-90ej pue ‘auoydaya ‘[rew 
ul ssuodsoruou Jo uosuedui0s Y (p661) ‘G'a ‘MnseT 9q 10 “ff ‘XO 


Cpb-vEPr “79 “JoYyoo]p uo saipnig fo ousnor 
“KOAING YYP2H SSIMS Ul UONdUINSUOD joYosTe payodal-jjas uo 
S$JO9JJOo S]JUDPUOdSal PUB SIOMIIAIDIU] ‘(100Z) “D ‘JeWD 19 “7J-"¢ ‘qaaH 


"UO}SUIAA, 
pur ‘yeyoury ‘YOH : ouoIOL “(Noidg “y'q 19 oquIepoH ‘q’A spy) 
‘QOUdIOJUT [LOSES JO suoNepuNo, Uy ‘syUSWTUIOD (1 L61) ‘+f “aleH 


"CS9-SE9 “ES “UONDIDOSSY 
/DONSIDIS uDIAaUp ay) fo jDUANOL *sy[NSo1 AJAINS Jo Aovindoe 9y} 
UO JOMIIAIUI BY) JO DdUaNTUT “(8S61) ‘S'A ‘SWRI 19 “HY ‘uosuepyy 


"PLE-IS€ ‘7 ‘BE IST ayi fo uyajjng ‘sKaAins pue snsudd Ul SIOUS 
jusWoInsesy (1961) “WW ‘Peusiog 19 “NAA ‘ZiIMUN “HW ‘uasuey 


“OUT “sUOS 
a AdTIAA UYOL : YIOX MON ‘TI ‘T [OA “Croayy pup spoyjayy Aaaing 
ajduvs (€S61) “OM “Mopeyy 19 “N'A ‘ZuIMINA “HW ‘uasuepy] 


‘99T-IS7 “OS “Ay4azavnG uoiudc 
aygnd ‘skaains auoydaya, poziyemusd ul sjdajjo JOMIIAIOJUI 
Sulurejdxa pue Zuunsesyy (9861) fT ‘AAPTISeI 19 “WY ‘SeAOIH 


“OUy ‘SUOS = AdTIAA UYOLS 
:YIOX MON ‘s7soD daaung pup ssosdsq ABAANG (6861) ‘WY ‘S2A0ID 


“S8-EL ‘A ‘sousynjg payddp ‘sKaains 3]dures omy 
WOY udye} AP[IQeUeA IJOMOIAIOIUI JO soydurexy] (966[) “‘DO'd ‘APIN 


‘OTI-6I1 “SZ 
‘ajanbua p sanbiuysay ‘oddei3 ap jajja,| & 19 UOTeIapUOd ze] ke SOT] 
asepuos ap uejd ap sjajjo so] inod ysry ap synuLoJ e] op s]9poul 
ap oseq e uoneoynsne (6661) ‘d “UtyeT 19 “S ‘JopeH “Ss “Je1qeD 


“TPOI-9I01 “6S “YONDIIOSSH [DINSHDIG UDIAAUp BY) 
fo jousnor ‘uoyeUIse sji pue souvuea ssuodsoy (7961) ‘d'I ‘132]]94 


‘€ ‘ON atydesouopy ‘]] Sua ‘suoday vary Apnyg ueMayo}eyseS 
‘UOHeZIUA, sed jeoipsyy Jo Apmg sayelogei[og [euonewoyu] 
OHM ‘a0ueLIeA JoMmatAlouI Jo Apmis YW ‘(€L61) fF “TOMI 


SUF 
‘gjanbua p sanbiuysay ‘sauiewop ap suosteseduios saj ins sjayyo 
SOS 19 INIMOIIAIOIUI,| IP DURLIRA BT (6661) LV ‘NOIS 19 “Cd ‘slaeq 


"8S-6E 
‘oz ‘dja120¢ yoavasay jayapyy ays fo jousnor ‘Addins apnyye ue ul 
$JOIJJ2 SULIO|SNID pUk IOMIIAIOJU] (7861) “A “Ioyoing 39 “| ‘sulyjoD 


‘Iaysoyory ‘Ad|IM\ “IDG 
daaing fo sistjouy (€00Z) (SpA) “fD “Jouuryg 19 “TY ‘sioqureyD 


INIMAIAIOUI,P S}|IGEUEA e| Ap UOHeJesdajul,| ap Jo UONIUYEP B| q : UIYe Je J981GeD 901 


andojejeo ne X-[00-ZI .N ‘epeued onbnsneis 


“SOI-€6 ‘S ‘Ssousynig fo ppudnor uDIjD4jsny ‘ssad01d JYseYydO}s 
SulkjJopun ue jo uonduinsse oy} WOY s[quonpap s}[nso1 sWIOS 
‘suoyejndod ayuy pue uoneumss oney (C961) “MUN Jomog 


ZE9-E09 “OUT ‘SUOS 7 ATI UYOL 
SIO MAN “(UIMAIL 'G 10 ZIeMYIS ‘N ‘oddiq ‘9g ‘Mnae] ap “7 
‘SUIT]OD "We ‘Iswag “q “SI9QAT "JT “spq) “ponG ssao01g puo 
juauaanspayy Aaaing sueq “eyep Addins Jo siskyeue ay} UO s}oaTya 
JOUD JUSUIDINSPIUI JO MIIADI V (L661) ‘ ‘UIMAIL 10 “g “IOWNsIg 


‘Q9I-8S1 69€ ‘08 “YOUDIIOSSPY )DIUSIDIS UDI4aUp 
ayy fo pouanor “sKaains xajduiod ul syusutuedxea souRLeA 
JOMATAIOIUT JO UBISAP JeUNdO (Sg6I) “T'S “SHS 19 “d'd “JOUlog 


“€T-9I “EL “UONDIDOSSy 
Joousyojs uvridaup ays fo jouanor ‘ajdureg sanig Adaing 
sw [euONneN ay} Jo sayio yoedum yysI9 soy Apnys souRLIeA 
JOMAIAIQUL UY (8/61) “WA “epleg 19 “AL “oop “7 “Aayteg 


“EPE-LEE ‘pl ‘youvasay Suijayavpy fo DUANOL “JIUeUA PUR SeIG 
JOMAIAIOUL JO SoINsRd (L161) ‘f ‘SUdAaIS 19 “TJ ‘AaTIeg “y'g “Iepleg 


a1ydeasorq 
“Q[OILe 199 OP I[CUISLIO UOISIIA PR] DIOT[IWIL 


UO Inb ‘sJyoONySUOS saiIeJUSUIUIOD Jd SUOTJSABsNs sins] ap 
SINDJVUTLULXA SI] 19 JOYO UI INDOVPaI J] SUOTOIOUIDI SNON, 


SJUIUIIID.1OWIIY 
Cf I jul d ) — 
(sre -s}en 
lw d 5 ur 2 zt 
i d+ ae d+ "d-"d= I < 
1=7 [=! [= 
mC 
dy) d d 
Ri ee a ud 2244. ™g—2d-] z 
ey 2 Gaye gle I I =a u 
a ral ae a 
dy 7 I Vf d 


nN 
~ 
= 
a T 
= ~< 
SA 
aj 
lA a 
N= ee 
7) 
= 
al T 
s ~< 
Ali 
TT 
ll = QX 
rey | SSS y 


™du +— Adu 
3 Y 1 ad 1=¥ I=! \I 
LCL Are GC 
: dy dy d , dy a #i 
1=¥ [=! [=d 
d 
rateKehunG 
My _ og S 4y 47 d 
veh « hh 
( 1) t 1=¥ (=! [=d Hi 
CEC 
dy a Bi 
SUOAP SnOU 


SOL 


J9AB 


‘gjeIgugs gyyesout | 
39 sonbuqesye suonjou souley9d juRSsTyN uq : aanaig 


{ 7 < | ty 2D) hel, Saff 


SIOTe “INOMOIIA 

-1guI [nas un.nb suoae,u snou ‘qq onbeyo suep ‘Ig 
‘TIVAR] OP IBIVYO SUIQUI PT SNO} JUO SMIMATAIO}UT Sa] onb 
}9 xnev3d sno} jUOs sptod say] Is yuoWanbrun ja Is gITe39 doAe 


tains d 
(1-3) d+(1-2)o+1e 


T> “"d>0 10 [> ?d>0 mog ‘9 wynsay 


‘sonbugasye suoneiodo sautepao yueNyda]Jo Ud 39 


i 1,0 apo cae 
ANF YU 3) y! 
RG Be KAKA 
GEBK Le Vee! #1 =d l= wieEe 
MIU RRO 0 wn COR Cd KiK 
: Id ay} ta TY eh 
(“4) Aye A 
(0) May, 


GS PSO 


‘sonbugssy|e suonel 


-9do soureys9 JueNjdaIJo Ud jINSUd,S }eI[NSaI J] GUeUAUTRYY 


600Z un! ‘ayanbua.p sanbiuyoe | 


j=! Wy [= Mu+ 
mc MU+ TIS)— 7M 4 ae iM! fae eee i Se 
vf 


a (mu + JOS) - ——- —_,— = 


md(T- ™u)-1- "d) T- aCe 
i 
fa — “fai 
“OANA 
4 {OS _ADtI 
er | aa 
itil i-az | 
. Le ils OSG AOD 
“d pos-iael 2) ap 
If — 
: Ro Mu \iS) | mut TOS 
"0 pos-ea' 1-2 | a Mt "fal 
T —_ 


SUOAR SNON “fp JOIJNSAY 


! ves ! Zi 
PRG iat Cunrs | ie) Lena Bey 
> = 


suous}qo snou ‘7 ynoy mod ¢ < 72 = 72 mog 


Bre tu" to WZ? ISs CWemeare 
> > 


suoua}qgo snou ‘7 no} mod m = ‘Mm mog 


(%e) ffar = (fe) fat 


suoua}qo snou 
‘1 yno} mod Q < ,O = 709 }@ 1 yno} Mod M = M og 


INBMAIABJU!,P BYIIGEUEA e| ap UONe}9Jd9}U!,| BP Jo UONIUYAP e| Aq : 


ango[e}e9 Ne X-[(00-ZI ,N ‘epeued onbysnejs 


‘(“e) fa = (Ce) ffat 
suoua}qo 
snou ‘Q= 79 olp-e-4so,9 “7 yno} mod ‘M = *" mog 


(%e) ffar = (fe) ffar 


suoud}qgo snou ‘7 no} Mod F/u = ‘u mog 


EUKd KE jew] 
mM un ious - mM tu !0 we 


= (“e) ffar — (%e) far 


SUOAR SNON ‘aAnadd 
! bg ! ! 
Silas pee! ENT Tat 
lat" fol jain Jou 
= 


Is yUauanbrun 49 Is (%e) for (‘e) ffat 
> 


SUOAP SNOU 


=e (DONG (ep ae p= © 


x! 
zm ik 


i) 


ARE eer Ohne 
= ‘Sp aae ("bp <D) = 6B Mod ‘¢ JojjNsay 


‘(4x “+ “e) ap uonouoj anb jue} ua (“e8) 

ap onbriumpiuesoy aytAvouo0o ek] ap snooap ayyesout, 7 
“xne3a ywOoS ‘U Sa] sno} Is NO 7 
yno} nod (| — ‘U) = ;O Is yuouonbrun ja Is ayes doae 


! ! ¥ 
pola < tu CS ae XK 
at? oe ae ree 
Oju" e+ Mure in Cue 

ake) 


Wye ye Ja}qeD vol 


andoyeye9 ne X-[00-ZI .N ‘epeueD onbnsneis 


! ! 


‘u 
[ss minke anbrydunt 


Il 
5S 


NUE “@xXdauod-nyos sa “9 onbsind 


Y 'y 
M(T =a) Ss Ca Tiers /o 


sioje ‘sjesau uoU JUOS sptod sa] sno} IS 


= “py sane ‘(1p © Np) = fe 
no “(‘e) ffa1 = “far seo jonbne “y ya 7 no} 
mod ‘m = ™m Is yuouronbrun ja Is ddTfLI9A Sa oTe39,T 
‘(%e) ffar ="d[p — (Ce) ul +15 “far suoae snou ‘ou0g 
‘MOMOIAIOVUI 1 oJ Inod uakour asepuos ap sprod 9g] 3s9 


auwigt: 


no 
‘y 39 2 yno} mod ‘Mm = “mM Is yUaUIONbIuM jo Is ayIPeB9 oA 


¥ ! ime 
¥! ! Sey 
eke Ucde 
SUOAR 
snou ‘ziemyos-Ayoned op syyesour.] yuesyyN uq : aanaiq 


= O.OSNC (Otay og 


ae 
mS 
it 
¥ 
wn 
c 


no ‘(‘e) fai s “fai °Z moynsay 


d+] = ('e) far 


suoua}qo 
snou ‘ziemyosg-Ayoned ap oyyesout | 39 | II[Nsar 9] yURSTTON 
uq “u/T= "™ ya Q < ™'d onb suosoddng : aumpyjos0D 


‘sonbugesye suoneiado souteyi9o JueNydajJo Ud 32 


eed 
Ewe ‘& Ae md + mee | 2 


=i et aay 8 


onb 39 


COL 


deme) 


ae 
Mask 4K 
Sins } 
mm Ck 

mM 

a om 


er 
anb juROU Ud 1U9IqGO;S }eI]NSEI 97] : aAnaig 
¥ ! 
(2 Ae ae) 
axe (“4) “eA 
G 


axouuy 


Mea = (“4) “ea 


aq Wie A 
Sion le "fat | wwynsay 


“sTWIOUSS 9UN JasI[eal ap anouLIEd 
‘uanbasuos sed 4a UOTTHUeYDE,p a]Ie} aed snjd oun 
Jo1a33ns jieLnod ‘mo} uos ke ‘e[dD “‘prepuRys spNUUO} vy Jed 
sanbipur xnao anb sajqrey snjd yuowiayou JUOs Inb xneqoys 
sjojjo sap Joras3ns ynod ajnunoy anou op uonesyyn,T 
‘g{qissod np amsaul ey suep [ego]s 19JJ2,[ OP NO MOMAIA 
-ID{UI P 1O}J2,] OP SUOTIUTJOP SaT[2ANOU SOU JdsI]N & 9}SISUOS 
Jasodoid suowumod snou anb uoneoyipoul ajnas ey { afoo 
-o}01d aidoid uos aiamnsmod ynod sajgnbua.p imaydaou09 
3g] ‘sojdnyynur suonsanb op yuourayieyn ne so1oosse sauratqoid 
Sa] asutaz0U09 Inb 20 uq ‘ajgnbua., ap uondaouos x 3a 
uoneoyiuryd ey mod sestjyn 4so ‘uejd ap no MIMOTAIOYUT,pP 
UvIPSUT joJJO,] aonb a]]9} ‘oeWIUIOS oINSOW 9UN jd ‘aTNUWIOF 
SUIQUI BI JURST[IN Ud sUONSoNb sasnaiquioU ap Inod sgynoyeo 
yuos urd ap nO MaMOIIAIQUI,p sjajjo say ‘anbyneid uq 
‘gyre InNjNy uN suep suoIdyey 
snou jUuop jueyodut oweaqoid un.p yise.s |] ‘added 
-PUI JO INIMIIAIOUI-PNUI SUOTJE[IUOD sap UOTeUIT}Sa | 
ap uonsonb ey] asestaAua sed suoae.u SNON ‘([QOZ SIAeG 
12 NODS *866] HouedueD 19 Yysrepeayonp].O “S661 HOSS 
}9 SIAeG ‘p66 MNIIT 9q 19 XO) sWIIQGoId ne jared 
Opowldl SUIWIOD SaSI]N 939 JUO XNedATUTI[NUI so[epour 
So] “(8661 HJouedured 10 ysreyeoyomny].CO “Z786I J9YyoINg 
12 SUTTTOD “SLOT JeIeg 39 e100y “AaTIEG “LL6] SUEAAIS 
yo Agleg ‘Iepleg ‘[96] ‘]]J09}9 UasueF{) syneurxodde 
sosodiodns suojjnueyos e suejd suteyiao sonbiydde.p 
gjqissod jsd |I stew ‘oplojIp 3sd ajayoa opuris ke 
adepuos sed ajyonbua oun suep a1ua38 99 ap urd un,p aiana 
ud ast ey ‘anbyeid uq ‘ourayqoid a] 1auIN0}UO0D ap UdAOUI 
UN }S9 “SIMOMIIAIOUI XNe JUOWIIITOJeITeB sagdajJe JUOS 
sso30L19}UI souUOsiod sa] [onba] suep ‘(9p6]) sIqouReYyepy 
red soy} o191u1d ey] nod gsodoid ‘sasodiadns suoynueyse 
e ued un.p uonesynn,] ‘soyeneds sadde1i3 ap sjayja soy 
IOAB SNPUOJUOD JUIANOS JUOS INIMIIAIOJUL,P s}ajja say anb 
sored ‘o[qissodwit ouIgur aIIOA ‘ayIOYJIp ys ‘odde13-enut 
J9  IMOMOIIAIOUI-BNUI = suOTR[aUOD sop  uoTeUIT}sa,] 
dyIssoo9u Inb ‘uonen[eAd,p o1ua3 99 ‘sajgnbua sasnoiquiou 
op sueq ‘oseuUO][NueYyos.p syyIqeueA ey onb opuvis 


600z un! ‘ayanbua,p sanbiuyoo | 


Snid JUSANOS }1k}9 INIMITAIOIUL,P dII[IGeUeA ve] anb g1e]su0D 
UO ([96[) []0930 UasuR}{ ‘o}IQUaZOUIOY,p saodInos xnap 
S3] jueredwiod ugq ‘jueyodun suIeWIOp 99 suep dYdIABYOOI 
RI Jossoiso1d ae} op oailessao9u aynop unone sues 3s 
[I ‘UONeUINSS 97399 op JUd}Ie) sattqnd sajome suteys0 aonb 
ualg ‘ajueyiodum ysa °d yo ™'d op aqeiy uoNeUNsa,T 
‘oyeneds 3a 
IMOIMIIAIOUI-VUI SUONL[IIOD Sap s9pI sUTe}LI9D 9UN SUOAR 
snou puenb ‘uondaouos ey ap ja uoneorrtueyd ey op adeja.] 
B so[qn ynoymMs jyuos ge] suep sadjuaseid saynuLOy 
so] ‘UonRIyHeys}sod red suonsau0d xne ja asuodai-uou 
RB] & snp sjayjo sop ajduroo sed yuouUdT} OU INIMOIAIOYUT ,P 
}9JJ9,P S9|NUUOJ sou Ip JUSWIANY ‘sat]IondaI JUOS saguUOp 
saq onb stoy oun ‘sed anbeyo e satoosse sjaai spiod say juaur 
-outessaoou sed yudjayjol ou sieur ‘ajgnbua.] ap uoldaou0s 
RT op odejoy e soyrueld yuos sjI reo ‘aseuuol[WUeYIa.p 
spiod sap yuouranbrun ajdui0s yuauuen sassodoid saynuuoj 
So] Suep SsasT]yN sptod say] ‘oyorse ImNy uN suep suTWIeXa 
elas jo jUeLOdUTT yUoWITeS9 4so Yq sawoalajjip suep 
JT[ICACY INAMIIAIO}UI UN NO Sed J] “Adf/ [has oun ke g}oaIye 
SO IMOMOIIAIOJUI UN NO sed J] DIOPIsSUOD sUOAR SNON 
‘saxo, duos sajgnbua,.p suonenjis sasioAIp suep 
soudoidde yuos mb maModIAIOIUI.pP sjayjo say 1ajai1ds9}UT .p 
J2 IMUYEp ep 3so sponse], op JHoolqo,] “prepueys onuoy 
eR] op uoneoydde oun sed onuajqo moyea ke] onb a qQIej 
snjd juowiayou ana elop weunod joenba, “mamodtAsjUI.p 
}29]J9,| 9p UoHonpal vy ve jospnq np amessao9u 4s9,U [I,nb 
opus snjd yred 9un,.p uoneoayye J Iayoussep weumod ejay 
“MAMOIAIOIUI | B SagIOSse SaguUOp sap syyenb ey] ap jalns 
ne suonednssoaid sap yUSWIITHNUI JayIOsns jreuMOd MaMaIA 
-IQUI.P 19}JO,] SP I9AIT9 Moyea ouUN ‘sed PUODIS 3] sue 
"JQYJO 399 op sJoQuUOS ne uOHUSNe,p JUSWUTeSYJNs Iapsi090e 
sed ou pelmod mMaMdIAINUI.p joJJo,.p prepurjs s[NULoJ 
PR] astjQn mnb sayanbua,p majdaou09 93] ‘seo Jarweid a] sueq 
‘UOneUITJSAaINS 3UN Ud UOHeUITSe-snos 93399 JOUUOJsUey 
jnad syeyeds uonrpau0s ev] JUoWIasNsLND ‘soyUdISYJIp JOS 
SINIMOIIAIO}UL SOP [IeARQ ap Sasreyd say puenb (qqq) Wes 
a]Iqeqoid saae aseuUOT[HUeYE,p uUeld uN.p sed 9s] sup 
MOMOIIAIOU!,P SJaJo Sa] JWOUIIAeIS IOWNsSd-snos jeLmMod 
a]]2 ‘ojduroxe Jeg ‘UOTeNjIS ek] UOTIS UONeUIT}Se-snos op no 
uoTJeUISaIMs op sug]qoid un Jasod jreLMOd MIMOdIAIOUT.p 
}9]J2,] Op prepueys synunoj ey onb 93e}suod suoAe snoN 


uOIsn}IUOD °*9 


‘sone .p 
12 “(S00Z) YT 19 YoU ‘Uoyey ‘(pOOT) J21qeH 190 UUAT 
(6661) UTyeT 39 JopeH “pqeH “(6861) ‘T0938 IJOUUTYS 


‘(L861) uRTeA “(9861) JouuTYS ‘(O861) ‘TOI Ie eULOA 
(S961) UST MOA ‘smajne xnoiquiou op Jed soguruexs 


919 JUO MIMOIAIOIUI,P sjajjo,p aouasqe,| us ueyd op joyjo.] 
ap saynuLIo} sa] ‘sonbuqesye suoyesado soureyss yUENDaIyJa 


andoyeyeo NV X-1(00-ZI .N ‘epeued enbysneys 


Ud Jd SISIOYS v [I,nb asepuos ap sptod say yuejdope ua ‘(ma} 
-PUIWIOUSp 9] SUBp u yUeUTLddns Ud oMp-e-}s9,9) a}sayTUeU 
g[[Inboo sun.p uoNsa09 sade ‘(Qg6]) ‘]]0o}2 eULIAA ap 
UOISSNOSIP eS SULP }JOH{ Jed sguUOp (py) UOYeNba,| op med 
B® onuajqo ang ynod snssap-1d aynuLioy ey aonb suojoN 


l=d ‘| ee; l=d 
mut OS 
d C 
sails gis obeait % 7 ee eee 
cM ou 7M ucu 
d al d 


Rynporas {fa ‘suontpuos sao sueq “y ‘1 ‘d 
yno} mod “m = mM 39 Q = ™d no seo 2] anp-e-ysa,9 
‘uoneispuod ap asse[o Pe] 3S9 Adf.] NO 39 IMaMaTIAIOIUT.p 
DTIQeueA sUNONL dJSTXd,U JI NO Sed a] aIapIsUOD ke IOP] 


‘(O861) JOO 19 BPULIA A, Op UOTSsNdsIp es sue : 9°¢ anbinwaYy 


(-(@)"4)"d+(-9)°d + T= ff 
SUOAP SNOU fiji = q SUNUIUWOSD 
ddn.P ate cone qqq ued un mog :¢°¢ anbinway 


‘g < 0” Is yuowionbrun 

yo Is Jutsod yso 2d — d = “'d onb 39 syne8au ane yuaAnod 

d onb isure 2d onb suojon (9-—De = (“{)IeA 
= v 

= ,O doar "ZW a[Qpoul np Jotfnonred seo un jso Inb 90 


(6.2) 0... 
6-9 


AIC yd 
Gi OE NGa OLN ule 


(47 end 7) AOD 


‘1 #1 39 dyno} nod 


(9-10 


‘ys 9 1 ‘d ynoj Inod 
A # 7x 32 1d ynoy maar 


j 


CMarAC DAL _ 
(aa aia AOD = 
‘Y ‘1 ‘d jno} Inod (0 ies De as (77) Ie A 


‘y ‘1d yoy mod g = (""4)7 


SUOAR Snou “Wanbgsuod Jeg 


(“x = ALT ‘ly —_ ee Oe 


‘ da 
'd AdN.] op seesousiut Sued u sop sundeyd 39 
d Jd. ep seesouayur souuosied “u sap sunoeyo nog 


INBMAIAIOU!_P YYJIGeWA e] ap UOHePudJayU!,| ap Ja UONIUYEP | aq : LIYE 39 1981GeH ZOL 


andojeye9 ne X-[00-ZI .N ‘epeued onbysneis 


‘d qdN.1 Suep J MaMEIAIOjUI | 
red saa8ouajut souuosied '%u sap ounoeyd mog 
: JUBAINS d[I9POU 9] 


suossIuyap snou ‘T >Q>(g “0)uTUI> ¢ onb yuesoddns uq 


‘Someutg SaguuOp sap Inod s]dums sjapoyp : ¢ ajdwaxy 


‘d qdA.J suep ,! INamararoyut,| Jed sogsouyul : yURAINS g]duloxa | e onbrdde_s ayja 

souuosiod “uw sop ounovyo 19 d qdf.| suep 7 Mamaia ‘g[dwioxe Jeg ‘d}XTU aTesUT] B[9pOUl Np eEp-ne pusj9,s dI]9 
-19juI,| ed segZoueyut souuosied “uw sap ounoeyo mog 19 syuisod yuaurajoms yustos °d ya ™d onb sed oyIssao9u ou 
fa uontuyap ey onb 1ausT]Nos op jURSsaI9UT 3S9 [I Guepusdad 


3 d be) 3 g be) 
’ 7 te 72 Ar 7 >g 72 SF 7 + 72 mg 
a c 


onb Jd1JLI9A ap 
J]IOVJ Sd [I ‘SUUOTUSUISNS 9}XTU SITBOUT] B[@pOU 9] Nog 


(¢ ‘1) = PJ Sed 9] Anod 7 ajduraxa,| ap 
(y 39 (j ‘(e seo say anod [eqoys jajsJa,[ & ANIMIIA1I}UI,p 39 ULL ap ‘UOleAQpUOd Vp SjaJJ9 SAP SAANLIII SUONGIIQUOD 7 INST 


000°0 ,070°0 
qu. ‘ Be 
B, RSGODS 277 es oS aca 


0100. 
* 10'0 
$1003 500'0 


SS 


0700 ¥ 3 Bed 000°0 
pe 700 
| Se hee \ : : 
| i a se \ Seats SESS 
SES 
| Vas s‘0 | SS ‘ s‘0 


em ne 
a 
a 


: 
ek esr a | yAeeray eats bas. Rae Ng iy 
XM =|) Nl ge ee aT oc a ee eee ee LL of 
joa Ya (y Je1 ‘f(y 121 “Ga (y 
000°0 0z0'0 000° 020'0 000°0 0z0°0 
TEGTN S000 7 tay nee band Wig WN 
ore > TT o10'0 
Ra 4 010° 
S100, 4 st00 i 
Se \. $00°0 
0200 \. paper 020° \. < 000" 
| el apo | Ng SN i sea 
iS ane : SS OSES OSS 
2 SSNS 
| "| Ss | 
“$0 SSS SS ‘0 
SS 
Ee ao ta NLS Been Rn | eee ee OT 
joa fa \o1 Ya GJ je1"“Ya G 
000°0 , 020° 000°0 0z0°0 000'0 020" 
. S100 9d wig $00°0 PEN, 2d 
\_010°0 010°0 7 ..,010°0 


x 


1 
oS t 
SSE | 
SN \ 


]o1 Ufa (e 


S100 .~ 


\ $00°0 
0200 \- 000°0 
H ; _—¥00 


| ~ tay Pe 


joi ‘ff (e 


wig $00°0 , as og 
0100,” ‘ 
na pio 
— 


ee 


oi “fa (e 


600z un! ‘ajyanbua.p sanbiuyoe] 


(GOON ~ "8 

y (£9 0)N ~ “J (29 0)N ~ 0 dane SyuepuadapuT 

JUITRID SoMOJegTe Sjayjo say aonb gsoddns yuo s]j ‘amd mauoa] 

ysa “3 yo yuowoAnoodsal ‘d Agr] B12 1 MOMOTAIUT] k 

snp samojyegye sjayjo soy wos “g “10 JeqoyZ y9YJ9,, sa 1 no 

as + AG ued wt Sk 

: JURAINS 

QXTU sTvOUT] a]9powl np Jatynonsed seo a] mod sinuU0j 
399 nud}qGO UO (G66) HOSS 39 sIAeq onb sud}ON 


no 
“(1—(v)"w)"d + 1 - (q)""4)°d +1 = fa 
suoae snou “qqq uejd un mog : p'¢ anbinway 
‘(p's onbrewial ey OA) 4 gqq ueyd un mod sarasqgo 
mao aonb puess snyjd ysa [egos yaya,] aonb arywsis ejag 


(I - (w)™4)"™"d + (I - (q)*"u)°?d + 1 = 
yr eee _u 

t nine asses up j=d 
Z ses 


suoud}qgo snou ‘9 je}[Ns91 Np dAnoid 
eR] suep sUUOp Ina onb yusUINsIe SsWOUT 3] JURSTTHN Uq 


9d + 2 fa 


! - 
i= dy - Ke jest? d+ 
antes 
z if 
=f 
op 

u wim’ < 

: eG Tile 


it 
=f 
[ ‘ulm < 
CNN Wie Li 
PRVVET Cry 
‘win Cu 
L 
‘SUOT}IPUOD $39 
suep ‘aonb 110A ap a]l9eF 189 [I “(POOT J91QBH 30 UAT) osSsRIO 
ev] ap sed juapuadap ou sasse]d sop InagquI.| e splod sap 
SQOUBLIVA Sa] 39 SQUUSAOU Say onb amp-e-}s9,9 ‘assed onbeys 
SUPP SSANRIAI sadUaNbay sp UONNQMISIp SUIQUI BT MOAR .P 
SUOUSILQUOD Sd] SNOU SIRU “MOMOIIAIOIUL x Ydf) sasseyo 
S23] Suep IJaLeA ap splod xne suoysuued snou [enbs] suep 


= jo 


Uu lid y) 


‘wutu 
Jatnomred seo 3] SUOISpIsUuoD : ¢°¢ anbinway 


INBMAIABAUI.P SYI|IGeURA e| Ap UONE}@JdajU!,| Bp Je UONIUYSP e| OQ : 


andojeje9 Ne X-100-ZI .N ‘epeued onbysneis 


SUOAR SNOU “INIMIIAIOUI x Yq] SUOIVIOJUI SASIBAIP Say Inod 
SOUIIU Sd] JUOS UONeISpUOd ap sasseIO sa] IS : 7'¢ anbuoway 


‘d = [ seo a] mod (76z ased ‘Ty ‘JOA 
‘€S6I) [O09 UssUe]{ SUepP sOUUOpP (/'¢) I[NUO] eT Issne 
SUOJOU SNOU “UOTXdUUOD 33399 SURG “IMOMOITAIOIUI UN & UCU 
-ouedde sayiun sap sumtu0d aseuUO][QURYD9,p Salepuodas 
SHUN So] 19}1v) SUOADP SnoU ‘aJLIRTIWIIS eT JOWUTYe WOANod 
Mog (OLE ased J ‘JOA “EG6T) MOPRP] 1° ZIMUNP “UasueY] 
suep a9uuop ({°¢) ued ap yajjJo,] ap aNUUO}J eT 39 apaogid 
mnb oyyeSoul.] op BIQUIOUI SWIOIxnep os, aud JUeTIWIS 
RP] JoUsI[NOS ap JUDIAUOD T] “qd onbeyo ke sayoayye JOS 
[leAey op ssIeyo SWIQU oUN jURAR SINIMIIAIOUI.P JIQuIOU 
SWIQUI J] IS Jeqo[S jOJJO,] SO DIQUIOU OWIOIxNop I] 


{1-Z)+(1-2)?d4 12 oe 


anb ajnosep II ‘9 yeynsal nq : [°¢ anbanway 


‘7 Ad/N.] Suep JUOS sMAMdIA 
-I9]UI SIOTWIap g saq anb jo [ Adf,] SUP JOS SMOMOIAIO}UI 
ayenb sap siatuaid vo soy onb onbipur (q‘v) = Py no 
‘(¢ ‘1) = PJ 90k ¢ NeaTqey np (Y 39 GJ “(e sed soy sed sgquuOp 
uonelspuod ap sjyoid syuaayip mod momorAsojUI-ejUl 
yo oddei3-enu suone[au0d ap SUOSTeUTqUIOS sasIoAIp mod 
[EqQO|S Ja]JO,] & MIMATAIQUI,.P sjayja sap jo sayeneds soddeis 
sop ‘uoNeigpuod re] ap Soaneial SUONGL|UOS sap sjouuoOIs 
-uswIpLy sonbryders sap suojuasgid snou ‘7 amsy el V 


He fee: 
(hss SyvaNees e hes I 
Bes Vio 
. Hf 
(T= (“q)°*u)-d +1 ca 
: IP = ole pe 


Wye 9 Ja1GeD OOL 


andojejeo ne X-[00-Z] oN ‘epeued anbnsnes 


Jed saguuop 
JUOS [eQo[S JoYJa,] B IMIMIIAIOIUT.P sjayjo sop jo soreneds 
soddei3 sap ‘uoneispuod ey] op SoAnRal SUOT|NGLUOD sa] 


4 


( 1=¥ [=! [=d | 1=¥ I=! [=d 
vd 4 < dy a 
; ba tied wig ; a et Id 
5 ¥ 1 jad Pgh =¥ l=! \1=d TDS 
mem ay xia ig 
é dy dF ad 4 dy oi i | 
I=¥ [=! [=d 
Udy "6 
é dy a Al 
7 = 


eletls 


d ay y 


anb Jaquowl suoAnod snoN 


‘USWMSIINe VC 


ae 
a( A) “am 6? § 
(“4) “re, pe 
i 9 ie Spec 
(“£) “rea 
no 
boli S PMMA Ea SASS 
HO” Ho as fo = CO ey, re, =ivfa 


3]2}0} DOURLIVA ap INa\eUINSA | BITUMOJ 
Jeqos yayjo,] red sosopuod uou uolHueyse,p suusAow 
eR] Inod gyq snos soueLeA ev] op UONeoTdnynu ey ‘onbisseyo 
uejd op joJa,] op uONeRseI9uss UN SUIWIOD dIDpPISUOd 


ang jnod 39 ‘MaMAIAIOUI,p s}ajjo sap 30 sayeneds soddeis 


tami Hiei dom, dysig9 
sap ‘ayesaul uoNeIapuod ey ap a}dwi09 yuay [eqoys 12]J9,T 


= (MIE UM D)AOD : iN 
: JUBAINS 


gjepour ne yodder sed gnojeo 4sa "ye, ‘snssap-I9 [eqo[s OJ’ 


100°0 I OV 
Z00°0 I Of 
£00°0 I OC 
r801 000 9681S 0r6°6Z Z60T §=00'0 ~— P88°LS 0r6 67 LITT €00'0 +=: 9E8'T8 0v6 67 St r000 I or 
0v0'0 I OV 
0£0°0 I Ot 
0200 I 0c 
980'1 €00'0 = S8r'0S 97162 760'I €00';0 ~—siTT€“9¢ 97167 SIT 7000 = Z19°6L 97167 St 0100 I or @ 
SLIO SCE » OP 
SZI‘0 I Of 
SLo'0 9990s: 0% 
8s0l L000 629°0r O78'TE 80 S000 = L@'8S O7STE poll 00d sg9d°cL OZSTE CT HMSTOCNE ECC OL OLN, 


6 Cc 0c 
LECT “GROO veer 108°T Heh ILEO €09°E 1081 O€TI ISs‘0- SSLZ 108'1 Sc vv v () 
I I OV 
Cc I Of 
€ I 0 
I9TT =8E00° = EL E'LI Ol Coll e000 Eel Ol LEV 1) VOC meet EC LC Ol Sc v I Ol (© 
Vv I OV 
€ I Ot 
C I 0c 
SEV 190°0- €l SE O8I'l — ¥S0'0- Srl SE SSli 20 O- S07 SL SG I I or © 
I I Ov 
I I Of 
I I 0c 
vert LIOO 9C Sl salt F S100" 6 Sl ISTI = O10°0- Iv Sl Sc I I ol @ 
0970 S60 SZ 
9LTO 8660 Sz 
SLEO © FEO SE 
Sell S00'0- 6 £0'6r 70761 €7IT 9000 68E8E tT 61 cell S00'0- — 87S*Lr COT 61 $C 66T.0> LEO nc SE 
“ff 2d my adn my dO “fa 2d my adn yaa! “*ff1 ~d m0) adn Mm ,) dU yar ! ! ! 
ara a Cys (pe si ‘ AS Sr oe . CQ) Cy)" ties ee ia u 
(1‘€) = WI (Z‘%) = FI (e‘1) = v1 
Z0°0 = J 32 
10°90 =""d anod “*ffar; ffa1 $ (7 ajdwiaxa) sasjguieaed ap suosieurquiod sanaisnyd anod sauuadoul anaMatAsazUl,p [IeAva) ap sadseyD 
¢ nesqeL 


66 600Z uinf ‘ajyanbus,p senbiuyoe | 


‘( lA Gelaiw “Mm) 


= M suep sonbyeipenb souloj sap sonel say wos 


=f J l= |=! [=d 
Msn € ROTOR 
f se ee mM adn 
= = u 
f \t=d x 1°" ped q) 
(lel oth 
Mula IR | a ee R 
d z hye El d 
}9 
=f 1=7 I=! l=d 
‘ula RRR 
if ay a d mM qui 
= = Uu 
a =f \ I=! [=d 1=7 \ 1=! I=d Cv) x 
Bee RK, . NLP Ce 
z he hi, Gi z ely, clip eel 
‘anno ug 


uorHueyos,| ap oyptey : “we =u 


‘d qdf.1 suep 

aseuUOTNULYDS,p soIUN.p siquiou sreadflig AGuank = fy 
‘1 INIMIIAIOUI,] B SagIaTJe d qd.] 

suep oseuuo][nueyos,.p sayun,p aiquiou : “wl!Z = "uy 


‘ou0qd 


‘f uoyergpuod ap asseyo e] e jueuapedde 
eseuuo]gueyse p soyun,p siquiou Pe lise = ‘wu 


‘f uonerlgpuod ap assejo ey e yueuspedde d qqn.] 
suep oseuUoTTHUeYSS,p sayUN.p sIquIoU eek = “wy 


‘f uoyeiopuod 
op ossejd eT e& JUeUsyedde 7 MOdMOIAIOUI] ke sagda]e 
d JdN.{ suep eeuuoynueyse,p sayun,p aiquiou : “wu 


suosstuljap 
snou ‘solorap sao Nog ‘UoyeIapuod ap sasse]d sa] JUSULIOJ 
BeCUdUI dP Saye} So] WUeYDS sed aT] ‘aseUdUI Np dIIIe} e] 
op yuowonbiun juspuadap jo “INSMATAIOUT,| OP 39 YdN.1 oP 
sjuepuodoput yUSANOs yUOS sprod say ‘presey ne sauUOTdI[9S 
juos aseusuI 99 suep suUOsIod oun jo aseUDW UN IS 

‘7000 < 7d Is (ye G mod 
QUIQUI Op 189 Ud TJ ‘9 < 2d Is (ae (e mod “ “far ap ayuaprud 
IMgea oun yso ffar ‘(¢ neaqe} a] MOA) Z s[dui9xa,] sueq 


(L- (“w(t - ™w) 


My = (“W)y 


Is JusuaNbiun 42 IS dIIp-k-}S9,9 


‘suostp ‘2d := ened 


qu 


Eons! 
CON Aen 


< (1- (“q)""4)°d +1 = “fog 
Is uouonbrun ya 1s {far > “ ‘fat 


‘9 < "'d mod ‘ouoq 


ango[ejeo ne K-[00-Z1 .N ‘epeued onbysnyeis 


ibrar td at 
i tgp Gp 


|r = ff - "far 


SUOAR SNOU ‘[e1OUIs Sed 3] SURC] : Ep anbiDway 


OL = “uy ‘Opt = (q)""u 
‘S00 = 7d ‘To = “d 1s ‘ffat-p = ffa1 onb Journos ap 
JULSSIIDIUT 4S9 |] “Opl[eA smofno} ysa ayI[eSout,] ‘“yuanbasuoo 
qed 49 [ & [esa ysd oU][eSoUl.] op dsiquioul oWerIxnap 
9] ‘SINIMOITAIOJUT S9T SNO} Inod SUIIUT RT 4S9 [TBAB ap adIeYO 
BP] IS “[- MOMOIAIOUI,p [IeAeQ ap sasieyo sap suusXoul 
RP] & [- MOMOIAIOUI.P [IeAeN op sadieyd sap seaiapuod 
suUsAOUI L] Op ONLI a] onb pues snjd jsa sayeneds saddess 
xne np ued op jaya is juowionbriun ja Is alp-e-1sa,9 


yu 


<({-(q)""w)°d +] = fag 


Is yuouronbrun ja 1s_ffa1>‘ffat 


_I- "a 
I-(y)"u 


‘0 < ™“'d mod ‘ouog 


({-(q)*"'u)?°d +] 


(I- =i) is wi “d= ffai— ‘fal 
adn, \ 9 = 
Pe (Gs rent I an 
[(0-(@)")°O+11- im 
SUOAB SNON 
T- (qu 
Toy) 2 


yo °d/ ™d sonel sa] ooAe ayUOUdNe 
OIeSOU | Sp SIQuIdW sUWTxNap a] oNb OA ap a[Idej 1S9 IT] 


(que d I-(q)""u ; 
Gu wl aa fal 


anb aylos ap 
(I= (q)""w)?d +1 1-(a)*"u Pe 
(eGo) 1 We ae 
= adn Id + 
= 0 =" + 1 = ‘fer 
‘ aonb 


su}ION ““‘ffa1 suep M MOLIZJUT sOIpUT,] JaquIO} sUOSSIE| 
snou “qgq ued un suosyyn snou onb stew ‘Q# 2d IS 


u is 
= “q sane (4g) = q no 8u"g < =(q) 
d 


jo 


1 [=d 


Uu 
d d d_d 
eas D D2DAP n(@ p))=V no lay! oRECa(Y) 


47 d 


suossIUuljaq : Zp anbanway 


INOMAIAIIU!,P 9P|IGEUA e| ap UOHe}9@Jd9ju!,| ap Ja UONUYEP e| aq : UIYeT 38 Je\qGeD 86 


andojeje9 ne X-[00-ZI .N ‘epeued snbnsneis 


‘(“q)*"u ev onbydde,s ameyuoul 
-W0d sUIQUI 9] a aia, CG KS JOAR Mago a, x 
= (“w)™u auuoj ey snos amiog,s sed juowloyeigues ynod 
ou ““u surut0s yno} “(“w) "uw aonb suojoN ‘tor yuonbrydde_s 
['Z onbieulal eT & So{NULIOJ soMeJUIWIUIOD saz sno} ‘ouog 
‘Z uOOaS eT B QUUOP “fal B a]QR[QUIAS }So B[NULIOJ 919d) 


cM 


C= Cw)" +1 = "far 
suoua}go snou ‘9g = °d IS: |p anbanway 


‘quea (“q)°"u onb sipury 
“Tdf] xne sINaMaIAIOUI sap UONEIDATJe,] 9p UONOUO] ua sed 


qui 


aurea ou (“w)"™u anb Issne suojON ‘| = 27 Is uawenbrun 


Jo Is gueda sone (“q)*"w 5 (“y)"w onb suoion 
1=¥ [=! [=d 
i & 
ae id, d ae 
SUS ifevid d, dm “ \ adn 
Tere iw dR ask 0) 
d 
MRR CK 
Nl dy 47 ef 
(G 
j9 
[=1 4 4 1a! dy 
id mu€ ial AS A ang 
q, I 1d, q, I 
OOAB 
I=¥ =! [=d 
Hts CRI 
¢ d d 
Lb ely eee da’ ‘j=d; du mM 
hie GRAS = Gq» (g9="4 
¥ Uu 
7 
i) 
[=¥ [=! [=d 
ie eX 
cM dmara plied 
Fit Lles Fig te id, idm M jul 
a eh ke 'u 
a oe ‘ (Cw) 4 
2 ‘dy dy] d 
c 
(4 
cyd id ed dy 
dy I 
ODDAB 
1=¥ I=! [=d 
RGR 
- d d Py d CO Biel ick: 
etd _ idm [To ‘1=1¢¢ 1dm pears! 
id, id aE ae (CC p))="V 
“MU 
raid, 
no 


= Se (“q)*u)?d + | yu 


= (“w)u d+ Lim ‘i ‘fat 


anb a[nosep II ‘¢ Jey[NsI1 NG 


26 


‘9 < “d Suourayjamiqep} “INOMOIAIOVUI 
-enut uoneuos ey ‘’d-d="™'d suossuyeq “qdf.P 
12 SIMOMIIAIOIUL,P I9UIQUIOD asSseIO-EHUI UONRIaLIOD x] 3S9 
d 19 qqf-enul uonejauoo ey ysa °d ‘sajapour soo sueq 


(a2 4a Ns 0 
Yeo “tars d ans od 


c 


- (4K YK) AOD ee 7 


el dad ASN Od 
re Paar | “1=1 fikores IS 72 
juoWIaTNe 0 

#4 (d=d is (09d > = (74 L)aog : W 


Cc 
— 6 — “ —_— 
A=4 (1=1 (d=d Is 72 


JUsUIDAT}OOdSsal ‘s}UBAINS SoTapOUI xnap sa] snos “4 

“ WN “ IW ‘ 
ap saoueueA say yuos ("4) “rea yo (“4) “IeA ‘snssap-19 
uoIssaidxa | sueq “xnesour sprod op 30 sajeneds soddeis 
ap aouasaid ey] JUSTFIUSIS M 3d S SIMOLIDJUT SOOIPUT So] NO 


(HO) FreKiet G., fh 
eae EE 
("4) “eA 


suossIuljaq 


= “4 


I=¥ [=! 1=d 7% 


mew OR 
d 


dy, d 


: uly UoNendod ap suusAoul ek] JoWIsa mod 

yins nb sa1apuod suusdAoul eT SUOSTTN snou ‘sed 30 sue 
‘d qdf.] suep eseuuojjnueyos _p soyun,p 
JIquIOU 9] aeaGAc = fae yogwn( “axe op tps 
=1'qg‘" ‘T=d) momarasgu ,,,! oy Jed sesousyu 
AdN sux? ®] Suep ouuosicd ,...¥ ral Inod axouu0s o8ep 
-uos ap sprod 9 32 uoneasasqo,, "4m ya "4A og ‘apeSou 
uoneiopuod ey] ap 19 (sayeneds saddeid ap s0ua}stxa,] orp-e 
-189,9) Fd. P 194 9,] IP MIMAIAIOUI Pp JOJJO,] Joredas suoyye 
snou ‘UBUdUIVYY (S861) SAYS 19 I9WIIIg Jed saI9pIsuoD 
919 B UONEMIS I[91 9UGQ “Yd 9uN,p snd suep oypieary 
OU MOMIIAIOUI UNONe.nb suosoddns snonN ‘sINdMOdIAIOUI 
xne JUSWdIIOyege sayoayye yWUOS sajgnbua say “qq enbeyo 
suep ‘no 19 Fdf oW9UI ke] sURp JUOWUUepUddopUT jUSTTIeEARy 
INIMIIAIOUI «uN.p snjd no uoneMIs ke] suoIapIsuod 
‘UOTDIIIS Op saesouI soyIqeqoid op jo sajeneds soddeis 
ap soUdso1d Ud MAMOIAIOUI.p soURURA ev] op daudoidde 

g[NWUOJ oUN suOUd}qo snou ‘UOTDas aUasoId eT Y 


sajeneds soddeis ja ajesoul uoneIgpUuog “Pp 


600z unl ‘ayanbua,.p sanbiuyos | 


ango[eywo Ne X-100-ZI1 .N ‘epeued anbnsneis 


eee 


1000 ~~ Ov 

z00o—O sd O€ 

€000 =I 0@ 
8660 ('v) ffa1 > (*8) ffai > “far > ffa1 6667667 0¢ o7* "F000 1 01 (y 


0roo Or 
0c00 = 0€ 
070.0 =~ 0z 

6660 (‘e) ffa1 > ('e) ffa1 > “far > ffar Ore ~—s E16 0€ SEPM OTOO FLT Ol (3 
SLI‘O €€el OF 
SAiOraen O€ 
S400 9990 07 

S10‘I (‘e) far > “ffai > ('e) ffar > ffar ozs Z8'IE O€ SZ, OSTO'O NEED HOI Gg 
S710 0S7‘0 Or 
S580. C€Ci0., We 


6 Got hY6 
8660 ('v) fa > ffar > (‘e) far > “for LTT OST 0€ St sot tae Ol (a 
I I Or 
4 I O€ 
€ I 0z 
S10‘I ('e) fai > (‘e) ffar > ffar > “far L‘97 Ol O€ SZ I 01 (p 
7 (aie OF 
€ I O€ 
Z Wo i207 
€ZO'I (‘e) ffa1 > ('e) ffa1 > ffai > “ffar S‘ZE SL 0€ SZ I I Ol (9 
I I Ov 
I I 0€ 
I i 07 
L00'1 (‘e) fai = ('e) far > ffar > “far 0€ SI 0€ SZ I igen 208 (q 
09710 Sr60 SZ 
9L7'0 8660 Sz 
SLEO 9EO'I SZ 
€00'1 “fat < (%8) ffai = ('8) fai = far SZ 0761 SZ S. 6600. T2Ol. HC (e 
“far | far ANIMAITAIIIUL,P SIIJJT Cea) eee ee On es 


10‘0 = "'d auuojos azgtusap e| suep $ (7 ajduiaxa) 
saajguivied ap suosieulquiod sinaisnjd anod anaMaIAsgjUl.p 3aJJa,p sajnw40J sap juauasse[D 


q hesqe Ll 
‘('e) far > (“e) ffar 
suOAR snou ‘anuTup 70 onbsind yo {far < “far ‘('e) fai > (8) far aonb anbydun inb 
suoae snou ‘uajuoursne ‘vu ya JM = onbsing (4 90 ‘ajuowisne ‘w jo JUONUTWIp {9 yo /m ‘anno ug 


G pee rent ne ‘ffai >“ fai ‘spuers JUSWISANRIAI JUOS me) sg] anbsing (2 
‘G suep onb yNed snjdysa 70 anbsin 


('e) fai >“ffar ‘anno uq  ‘(‘e) a1 > ('e) ffa1 ‘('e) ffa1 > (“e) ffa1 suoae 
SUOAR snou ‘ajuaWIsNe ~O onbsind 4a ffar<"ffar snou ‘ajuawisne ‘u anb yo onutuIp /o+ /m anbsing 
suoAe snou 4uajyuousne ‘u 4a ™ onbsing (3 ffar >"ffai ‘spueis yUSWIOANeIaI JOS 70 sa] anbsing (Pp 
‘(“e) {fat > ('e) {fat suoae snou ‘xnap ‘(“e) ffa1 > ('e) far suoae snou ‘xnap 
sno} jugjuauisne ‘u ja /o onbsind : {fai < “far sno} juajuowisne ‘u ja /o + /m onbsind ‘anno ug 
anb anbydun juajuowisne ‘uw yo (m onb yey ay G ffar>" fat ‘spueis yuauaayeyas jos 79 saj onbsing (9 


INAMAIMAJU!,P 9Y[IGEUEA e| ap UOHeJesdayul,| ap J UOUYEP e| 9q : UIYeT Jo JEIGeD 96 


anZo[eivo nev X-[00-ZI .N ‘epeued enbuysyes 


‘(°e) ffat 

= ('v) far onb onbydun .m-2= 09 ‘agno ug 
ffar>" far ‘spuess yUOUIBANeIEI JUOS 70 Saf enbsing (q 

‘0 < © amb cored ‘sane 

sa] sno} onb nad snyjd ysa “Yar ‘anno uq *(‘e) far 


= ('v) far = ffat ‘xne89 yuos ‘u saz sno} onbsing (e 
‘syUdIaTjIp spyosd yiny soy yueUa}UTeU 
suonbydxo snon usa mu’ ‘ojduiexay sueq 


‘(e) ffar s “far 
19 ('e) ffa1 Ss ffa1 onb yey np ayduioo yuo uo] Is (“B) {far 
“(far ‘('e) ffar ‘ffa1 yuenbasuoo sed 4a (‘e)u ““u 
‘('e)™u ““u ap satqissod syuaulasse]o say sno} jUaUOUT 
suosteulquiod saqne xIS Say “9 x» “m anb asoddns 
ouWaIxnop ey] “xnesoul juos sprod saz onb sieur ‘sayeso 
Juos ‘u ap sinaea sa] aonb asoddns asgiuaid ey (9 “mM “u) 
ap sayUsIa}JIP suOsTeUTqUIOD yINY sUUOp Z NeIzqQe} IT] 


‘7 ajdwaxgq 


“xne3o sed juos ou sprod say janbay] nod 
INIMIIAIOUT UN SUTOUT NB d}SIX9 [I,S Ip-k-1S9.9 ‘Q < JOG 
Is (¢) aNWUOJ ey JosI[WN suoanod snon : ¢'¢ anbinway 


‘(e 7 oJduiaxa_| suep soup }S9 INIMATAIOVUI, p 
[IeAey op osieyo oulguI ey ("2 ap ajUessIoIO UOTOUOJ 
aun sa uoIssaldxa s1QIWIap 91390 ‘juRJsUCD "4 nod) 
(U0 +1)/"2 3 “Ue sUNUIUIOD INaMaIAIOIUT,P [IRAP 
OP IIeYS L] ap aJULSSIOIO UOT}OUO} 9UN sa | /far— "fat | 
‘anno uq “fat INAMOTAIOIUT.P [291 JofJa,] op ayuaprud 
Ingjea oun yso ffai ‘ouog (0 # JOS oanb juesoddns us) 


<a = fin — "fo 


VV 


SUOAB SNOU ‘SINIMITAIOJUT Sd] 
SNO} Nod sUIIUT BT ISO [IeAR IP 9BIEYO kB] IS : p'¢ anbanway 


"SaTe39 JUOS SMIMAIIAIIUI SAP 

[le@AeQ op sosieyo soy Is JUoWOnbrun 3a Is sayuayeAtnbs jos 

So[NULIO} xnop sol “qqq uejd un mod “wowwepuq ‘fai 

op sepow ied ogjsIsse uonRsIdiajuI.p sed suome.,U snou 

‘MOMOIAIOJUL UN SUTOUT Ne mod yuaLIeA a8epuUOs ap sprod so] 

Is ‘a]duioxe eg ‘fai ap ajapour sed ogysisse uoneyaidiayut 

oun jUusWoMessooou sed UMNO} OU a[e3d INAMOIAIOUT,P 
[leaky op oszeyo oun ‘qqq uou urd un mod ‘ouog 

"Y 2 1 NO} 

mod ‘uf /[ x» “ 12 “= ™mM Is yuowonbrun ja Is ‘fai 

= “ffal a1p-e-}sa,9 ‘y¥ 39 1 NOY Mod J /{[= Muley / mu 

19 (0 = JOS ‘quseamba osigtueur ap 10) m= ™™ 
Is Juowionbrun jo 1s ffar = “fat suoae snou SueuaquIeyy 

“UONeNIIS d][9} ouN,p 

souasgid us sauTUOs snou ‘(p ja (9 ‘7 afduiexa,| sueq “far 


S6 


Q]JO91 INIMIIAIOUT,P IOULLIVA P] Op d]UNprud Maye oun }10s 
ffat anb 20 & apuayye snou suoanod snou ‘9 # JOS enb}e 
JUDWIDSIOAUT Jd BBepUOsS ap suddow sprod s}yad ap eB d9TD0ssE 
a9 & IOULPUD} B I9AI[I INIMOIAIOIUI.P IdIVYO UN IS 


yl 


VR Wisse 
TOSS aul IR Is yuouonbiun ja is QS ffai— “fat 
I 


anb onbrydun 
(7) 0< “dO Ig qda sed jso,u ued 2 Is amp-g-jsa,9 
‘0 # JOS Is (Z) Jestyyn suoanod snon : ¢'¢ anbavuay 


‘1 4no} Inod 7 /u = ‘u Is yUoWONbrun 4a Is 
ayesa cone Q = f/far — “ffa1 aonb sasued e ouuop zIeMmyos 
-Ayoned op ayyjesout., op uoneodde oun,nb suojoy 


ee ee 
Uu I ae Ty << wy = ffal — “fal 
I 


SUOAR SnoU “A qq sueyd soy Mog ‘UOTReNIS 33N0} 
suep ([) 9[NWUOF eT JosIWN suoAnod snon : 7'¢ anbuvuay 


TOS A+ 
qu mi (6 Ae 
(¢) d I f 7 |=! a yi 


u Uu 


ss el Po nb OS OBA FM) 
(Z) “| 0s. 7M ofr Rae = 
I 


Uu u 


(1) 9105, MM of ZR 
[ 


Hai—" fat 


(p Je][NSOI J] MOA) onb JaMuOW suOANOd snoN 


‘Ja]duI09 uoT]HULYSE | SUep adepUOs ap 
splod sap uojeura ap yuaroyjaoo a ‘Mm / us os/ = 

*9[P10} SUTIqeueA 
RB] B BSepUOS ap splod sap IMIMOIAIOJUI-CHUI DTIGVUeA Pe] 

dP dANLISI UONQLHUODS eT Ap IMd}eoIpul UN ‘7FOS/JIOS = 
‘9sepuos 
ap splod sap saued sap a]e}0} oUIWIOS BT ‘799+ FOS =LOS 
‘adepuos op splod sap MIMIIAIOJUI-e.UT 
oul <= (u—"a) I< Tk= 108 
‘adepuos ap splod sap smaMorA 
“IOUT aUS SPLed sep suNUOS eT “(mM — m)'ul'"< =998 


SoUeD Sop QUIUWIOS PB] 


6002 uinf ‘ayanbua,p sanbiuyos |, 


‘SMOMOIIAIOIUI SOP s[quIasud | Inod uaXkour 
asepuos ap splod ay “mM uyuli = MMI ie uta 


SUOSSIUTJOP 
‘ejao nog ‘ffa1 jo “ffa1 agua soudlayjIp ey juanbiydxo 
smajovj sjonb MOA op JaAessa suoT[e snou 4ueUaUTeE/| 


(ie) ffar = (Se) ffar = “far 


1) 


SUOAR Snou [enbg] mod 4 qq ueyd un jsa a1ua3 99 ap asepuos 
op uejd op ajdwiexa up ‘suU0p moaMoatAIaJUI un mod 
SOUIQUI Sd] SNO} JUOS a3epuOS ap spiod sa] Is yUsWenbrun 
39 IS SaquaTeAINbs yUOS (<8) fat yo “fat saynuuoj say ‘ouog 


Sy CON (dna fot pe Se 
* * * * 


‘(“e) ffa1 = “fat 


seo jonbne ‘y Jo 7 no} 

mod ‘™ = mM Is yuoulanbrun ja Is ddIfLI9A 3S9 ayI[e39,J 
‘qquopnid Mamata 
-IQUI,p soueLIvA aun sUTUIOD ajoI1dIajuT aya yned (‘k) /fa7 
‘uog ‘s[nu jUOS “0 sa] sno} Is yusuanbiun jo Is (‘) ffa7 
= “fai < ™"d mod ‘apgogid nb uonenbs, sueq 


y 1 
mK 
eit of 
—= 
mm < 
™ 


DINE (2 et 8 op yas Ag 


no 
‘(4e) ffar s “fan 
anb g]noosep [I ‘oxouue ua 7 yey]NsaI nq : ['¢ anbunway 


T= 'p'X ovae (1 - (8) "u)"d + 1 = “far ounioy 
RB] snos am199,s sed ynad ou “Yfai ‘fe1Qued ua nb suojoN 


1 ! 
fowhs wt 
1 = ree 
yg! 
mur’ 
no 
“(I M st) d a I ft “fat 


anb Jayuou ynad uC 


anBoje}ed NV X-100-ZI1 .N ‘epeueD onbysneis 


‘MIMIIAIOJUT ,,,.,,/ 9[ INod adepuos ap spiod sap soueLeA ey] 
Zu : LV lar ay Wa 
Mh — MUN U/[= ,O 12 NIMAIAIUL ,,,,,1 a] INod uakow 


asepuos op spiod ay “ml? ‘u/s, = ‘mM suossiuljaq 
“aXoUUP Ud | JeI[NSaI 9] MOA, 


‘oe 
EIR ears 


ae 


[+a 


(C6) Mien» 


SS Secs Cl al 
("4) “eA ii 


Jed a9uuop jso s[apow Jed d9}SIsse INDMITAIOUI, p 
soueleA ke] ap sjnunoy e] 19 (TL61 RH * C961 
JoMolg IIOA) o1uly UONe[ndod op suuaXou ev] ap asiyyn 
yuouMUonbey Inayeuse un yso "MIT MAMI ="4 
sgiopuod suuaKowl ev] ‘UONeNyIS 91399 SURG “INIMOIAIO}UI 
ou! 2 Jed sgsouojur ouuosiod ,,.¥ eB] e& g100sse 
asepuos op spiod aj “" jog “xne8eur juos spriod so] 
NO UoHENIIs ek] SUOUTUIeXe snouU “UOT}Oas ajUasgid eT 


sajeyeds soddeis sues a[es9ul uoneAIpUuOg “¢ 


(fe) syuaagyyip 


anod *d ap uoyouoy ua ('e) ffa1 ap sanbirydesy | aaunsry 


ulayd yredy : Dd INIMAIAIOU] 
Q][QuIod : g INaMdIAIOU] 
NdWOLIzUI 1} 2 YW INIMITAIOU] 


(‘e) ffa1 


‘0% QO] UOIIAUD Pp UOTBUITSO-snos 
gun Jouuop yieunod ffa1 onb Ja}0U op juessaIg]UI 4s9 
[I ‘| onal e] JURUTUIEXS UT “MOMOIAIOUI-EQUI UOTLI9HOO P| 
Jo NIMIIAIOJUT,P [IAL Op Sadieyo sap so19puod suusAoul 
B] SIIp-k-}S9,9 ‘sJUSNIJUI SINd}OeJ XNOP SOP SUOSTRUTQUIOD So] 
sajno} nod sjapoul Jed s9)SISSe INOMOIIAIOJUI,P DOURLILA Op 
g[NuUo} aou ap 30 ( ffa7 ‘omp-e-}S9,9) plepue}s a[NULIOJ P| 
dP UdAOU NL SdNUd}GO INIMIIAIIIUI ,P IOULLIBA P| Op SINI|eA 


INPMAINOJU!,P S}JIGeUEA e| ap UOePudsajul,| ap Jo UONUYEP e| aq : WIYeT Je Ja1GeD v6 


anZoyeyewo ne X-[00-Z1 .N ‘epeued onbysneis 


so] ouuop | amdy ey “ffar=(° 'e) far z (4 ''e) ffar 
z("''e) ffa1 onb oanbydxa mb 20 ‘axaauoo-myos 
so ('e) {far uonouoy eT ‘9 < “'d mog “yuowoansedsa1 
‘) 19 g ‘V seo xne juepuodsau0d sfa=powl Jed oag\sIsse 
MOMOIAIOJUT,P DOULLIVA ap apnuTIOJ eT “('e) far yUoUsISEp 


mb ffar = (?''e) ffar yo (7 'e) ffar “("'e) far tos 


6L 08 ZEl ('e) uy 
06L 06L 06L u 
6L 88 Z0@ 01 
6. 88 91 6 
6L 88 O€1 g 
6L 88 001 L 
6L 88 bl 9 
6L OL ZS ¢ 
6L OL re F 
6L OL 0Z € 
6L OL Ol Z 
6L OL t I 
& iO? Si Nea FOr S(t SolfanMemeayS 
SAINI MITA1I}UT 


Sop [IBABI} OP SAd1VYD SIP [YOIg 
(1 2du9Xx9) Sand MatA193UI 
XN& [IARI] Op SadivYyd sap SajUdIIJIP suOTe IIE sory 
T neo[qe 


“SINIMOTAIOJUT XNv Sayed [TRARY op sasdreyo 
ap uonEpaye,| e@ puodsauo0s (D sed 9] 10 ¢°6 = adAj-pIe09 
un dade siqiinba onbsaid ysa (q seo a] ‘¢°gg = adAj-yIe99 
uN 9dAP JIGeURA snyid ey] [IeAeI] Op SasIeYS sop UuOT}ROAYe | 
ajuasaidal (Y S¥9 IJ “N6L = U DdAB SUNDRY ‘SINOMOTAIOIUT 
XIp Inod sqyuolajjIp [leary op sosieyd op suoneRayye 
SION SUOIBpIsSuOD snou ‘][ Nesfqe} a sueq : | adwaxy 


"SINIMOTAIOJUI SOP [IBA] Op SadIeyo ap sypod sIOAIp 
mod sa1ayjip yuoreunod /fa1 yo ('e) {far ajjenbe, suep 
oso ek] SUOQUOWUSp snou 4ins Inb ajdwioxa,, sueq 


Har < (e) ffar 
‘xqpeo sed -juos: ou lw’ soy onb a’ pi< °-d 
Is ‘onb | ye}[NsaI Np aMe][OION np snovap I] : 7°Z anbanway 


sIoje 


(O86) ‘1199 19 BULA A Op JURINISIP JOH Op oomue | 
suep souuop (¢y) ued ap syajjo sap ajnuuoj ey 39 ('e) far 
INIMOIAIOUL,P ITIGeURA PR] IP [NULIOJ ek] ANUS dILIeTIUIS 
B] IOUSI[NOS op jURSSdID}UI jsd JI ‘sIqyinba uou sed J] 
mog “WW op Jotynonzed seo un jsa nb 9}x1ur arteauT] ajapour 
un juesyyn usa f/far ap ajapour sed agqsisse uoneoynsnf 
aun guuop & (7961) Ysry “siqifinbs seo 2] Mog “MaMatAIOIUI 
anbeyo nod swi9ur ve] 1S9 [IeAeI] ap adieyo P|] Is yUowWonbrun 
}9 IS OIIp-B-}sa.9 ‘2 yno} Nod 7 /u = ‘u Is yuowenbiun 
9 Is fat = ('e) fai < “d mod onb ‘oxouuey e 
QuUOp ‘| Je}[NSO1 NP se][OIOD Np ajnod~ap I] : ['7 anbanway 


‘INIMOIIAIOJUT SUTQW UN 
Jed sat[[landai SUORAIASGO Sap a}I9UaBOWI0Y, | }Nponut “py 
gjepowl a, “py aJapour ne yuswaenuOs ‘anb suojOoN 


€6 


‘JUSUTOI}Ne 0 
Paar Rev M CMe Sune ba A Gk (00) Cy tra 11 
“y= ¥y"r=Tis 7? 

JuowIoINe | ni (me )A05 W 
“Y= ¥ P= TIS 70 


‘yuaUIdATdedsal ‘SJURAINS S3[9POUI xNap Saj snos 4 ap 

c l 
ssoueueA sa] yuos (4) “eA yo (4) “eA ‘aguuoNuoUsns 
ginuoy ey sueq “uw f'u =p cone) “(0 Tip) ean 


(4) “ren 
COMIN 

suoua}qo snou ‘oxouue | 
B guUOp | JeINsQI a] JuestTNN ug ~(°e) far = ffar 19 J/T 
=p po Sp y= Me Tose (le) r= 2 SIDPSUOION 
‘MIMOIAIOUI,| Op [IeAeN op sasieyd soap sasopuod uou 


guuckoul e] sa “uno ‘d(T — uw) + | = fat omp-e-3sa,9 


‘MOMOIIAIOJUL .P JaJJo,p onbissepo anuuoj ey op s[apour sed 
ag\sisse uoNeolysnf oun JouUOp Jed sUOIO.USUTUIOS SNON 
‘((‘0 © “'p) = & Jo MamoaIAIOyUI 
sug? MP [IeAeQ op adreyo ej e gnbidde omeniqre spiod 
uN 3Sa ‘D NO “MIMIIAIOJULP [IVAB Op SadIeyo sop so1opuod 
suuasow sun “u'o ir? = (e)"'u 38 agigpuod uou uolyH 
-ueyo9.p ouuasow ey “ANI w/t = A ‘uoynueyog,p 
ayeqoy ayer ey “uly =u suossmyeq (Cu T= 4° 7 
“= 71) IMOMOIAIOUI 4,4 o[ Jed sesoiojut ouuosied 


sust Fe] ep saidne onuajqo uonearasgo], "f yog 


mir — ('e)“u]+]= = ('e) ffa1 


sajeneds soddeis suvs Yqy ue[d ‘7 


‘oxouur | B Sagjuasaid yUOS sanbruyos} s}e][NSoI S23] SNO} 
ap saanaid sa] ‘sonuuos onb Isule sajyaiostp saguuop so] 
Inod opr[va UONR[PL09 op [elouss JJopouwl un ke 19 Yqq uou 
ueyd un B (S66[) NODS 39 sIAeq Jed JusUIAINaLID}Ue agsodoid 
Q]NULOJ ouUN suUOSTTeIQUaS SNON ‘oallessadou jIR}a [eqo]s 
}9JJ9,] Op INULIOF 9]]9} oun,nb suuONUsU ev saspoH ‘(O861) 
YslepRayoIINy,.C 39 HOOS “CULIDA, Op IOI] 9p UOIssndsIp 
BS SUR] ‘O[R}O} DOULLIVA BT SIINPdl ap SUSAOUL SIOAIP Op SIA 
-nosep ke] JoypIOey nod uoNIsodurosap 39d “MaMdIAIOUT | 
e jo soddeis xne ‘uoneiopuod ey & snp sjajya ud [eqoys jaya 
J39 Op UOTRINUSA a]J9q oUN suOjUassId snOoN ‘UOnROTFIURTd 
eR] op odeja,| & UOT[HURYIE,p IIe} BP] Op 9}9exX9 UOT}BUTULIA}Op 
aun sUUOP [eQo[s j9}J9,] IP [NULIOJ eT “INIMOIAIOVUI,| Op 
}9 sajeneds soddeis sap ‘uoneigpuod ev] ap sautquiod syajjo 
SOP ULINSSI JOJJO,] Op suoyndsIp snou ‘¢ uoIdeS FT VW 

“MOMATAIOIUT, p 
S]Iqvuea el op souepiodun,.| e@ juenb sume assnej oun 
yUDWZTNUI IauUOp yned far ‘sayeneds sadde1iB oaae saxajd 
-W09 sajgnbua say mod ‘ouoq “INdMAIIAIOUI.P d]IQeLeA 
R] JOUITSOMS B DdURpUd} JUSUaTeIOUES Be {fal J{NULIO} Pe] onb 
suousjnos snou ‘soyeneds sadde1s ap seo a] sueq ‘soyeneds 


600z uinl ‘ayanbua.p sanbiuyoe | 


soddei3 op jo ajesaut uoneiopuod op souasaid ua maMoatA 
-I9JUI P SI[IQeURA PR] SP UOTTUTJap a]JaAnou oun suojUaseid 
snou “p wondas ke] Y (€0OT JOUUTYS 32 sloquLYD ‘1661 
UIMATL 19 JWG - 6861 WWE 19 YOH “UUTYS * Hg6I 
ODS }9 ORY ‘SaNe oUS “IIOA) aINjyeIOAT] eT SUP S}LIOSp 912 
JUO S9XBUUOD s[a1dIso] sowatqoid say onb isure ‘sanbysyeis 
soinpgodoid sasIoAIp & SUOT}ODLIOD op yodde,] 19 sayanbua 
S3]_ IMs sYoIOYyOI eR] op saysiteloads xnaiquiou ap sed 
NUUODI 9}9 & We 9 “SINAMOIAIO\UI sap soddes3 ud astute] 
onb 313) suIgUI ne ‘prepuR}s onbysye}s souaJaJUI,p Sapoyaul 
So] Suep ose JUDUTWONbay (SsanquysIp yuoWanbyuapt 3a 
Sajuepuodgpur saqrun) pit asayjodAy,] aor ayeneds yt9u9s 
-OWOY 9119 ‘aJonbua,p soguUOp Sap dUID}UI 9}19UdBOUIOY, | 
ywor1sse «Inb 30 ‘aqeiquias anbruouo0dg0100s ~—a}xayU09 
Ind] op asnevd ke ‘sapnyye SOWIQUI So] MOAR ke soURpUa} 
yuo ojeneds oddeis swigui sun suep judaarA Inb sauuosied 
So9J ‘soquUONs[9s jUoOsS sayooidder juswajeneds xnot] 
Sop SUep JURAIA SouUOSIad sap ‘saIdap sinatsnjd ev saddexs ua 
oseuuo][nueyse.p uejd un snog ‘ajduns anojeaye aseuuoy[y 
-uevyo9,] onb osnaynod sUIOU sATRUIA]e dUN SWIUWIOD oIOp 
-ISUOD }So SgIsop smMotsnid e sodde1s usa sseuUOT]HURYD9,p 
uejd un ‘ajonbua.p yeqo]s jnod np uOTONpal ev] no jeI9Uas 
uonendod op aysiza1 un,p souasqe,.| anb saya} ‘argioueuly 
}9 B[OUUONesTUeZIO SINjeU SP SUOSTeI SasIOAIp Inod ‘aTjayoe 
opueis e asepuos sed sajgnbua sasnaiquiou op sueq 

“UOT[HULYSI | Ip a]]1e} VP] Ins soUNpTOUT ouN eine UOTROTFIURTd 
OP SINULIOJ Sap UOTBIFIPOW PJ ‘YLOIOIMS AC] “INIMAITAIOIUT ,P 
S]IQeueA ev] op sjoQnUOD ne gs[qeuUOsIe1 snjd jospnq 
un sJaaye.p ouop jouuod 39 IMAMOIIAIOUI.P dI]IGeUeA 
PR] op ajoexa snjd uonenjead oun jUMOJ s[NULIOJ AQON 
“MOMOIIAIOUI P sjajo sop sjonuos ne souvyodu.p nad 
don Jopiosoe yuoreunod s[NUUOJ 3799 yUasTTQN Nb sajyanb 
-ud,p simajyd3ou09 saz ‘uanbasuos sed ‘49a uOTeWIsa-snos 
op owigyqoid un Jasod yreunod /fa1 no suonengis soureyss 
QUUOTUSUL JUSUTITESS SUOAR SNON “INIMAIIAIOIUT,P S}ITIGVUeA 
R] Op JONUOS ne ailessao9U JUDUTIVIA 3S9,U [I.Nb juepyodut 
snd jaspnq un juowsfqe[quiasieia viojaye ffai oasipyn 
mnb sajygnbua.p inajydaou0s 3] ‘uawastoaid snj{gq ‘ajenbua.p 
s}nod sap UOHONpesl ey] sWII0U09 Mb 99 ua anbiyeid j919)UT 
un jUO S}e\NSoI Sdd ‘SaTNULIOJ xnap sao aud sOUdIAFIP 
PR] op UuIsUIO.] eB sIMa}OVJ saz Issne suoasesasp snon 
‘gjapoul red sajsisse aynuuoy ey] ed aguUOp IMgyeA ke] onb 
gagj2 snjd sinofno} ysa ffa1 aonb suoquouwl snou ‘asepuos 
ap spiod surgur aj sno} sed yuo.u sgjgnbua saz jenboy 
INod IMIMIIAINUT UN SUIOW Ne d}SIX9 [IS ‘SINOMOIAIOJUT 
Sop sojea [IeAeQ op sasIeyo op seo uq “//a7 .p sJgpowi sed 
agisisse uonRIdIgjUI aun JUdUTOMIessao9U sed yURIeS OU 
MIMIIAIOU _P [IeARI] ap sasieyo sop oje3sa uoNNedss oun 
‘qdq ued ne yuowomenuos ‘anb JausI[nos sp JuURSsoJ9}UT 
}So [J] “INeMOTAIOVUI.. mod unurwos spiod np sued ne 
J]JouUOTOdOId JUSWIDSIOAUT JS NIMIIAIOJUT,| OP [TRA op 
adreyo ey anb 4a asepuos op sprod suIQUI 9] JUO NIMOIIAIOJUI 


ango[eje9 ne X-[00-ZI .N ‘epeuerg onbnsneis 


suiguI un Jed sassoliajuI souuosiod sa] Is yUoWonbrun 
yo Is {fai .p sJopowl Jed agqsisse uoTR}oId19}UT dUN sUOUa}qO 
snoN ‘soyeneds soddeis sues seu ‘ayesour uoneigpuod 
9un.p 9dyl[Iqissod ev] suoutwexa snou ‘¢ uOT}OeS EF] VY 
‘arromaiqey 
g[duis suushoul ke] & MMOIIAIOUI.p salapuod suusXoul 
[fea] op osseyo ev] yUeNNSQns us /fa1 op med e anuajygo 
ang ned MIMIIAIOUI.P Ja]J2,] ap soudoidde sjnuvoy sun 
‘UONENIIS 97399 SUBC] “INIMOIIAIOJUI.P dT[IQeUA ke] Ja;O.NUOD 
Mod oMessad9U JUSUIIT[IEI JS9,U ]I.Nb YoOJja,p suloul edz 
fat asin nb saygnbua.p majdaouos a] ‘ouog ‘aantsod jsa 
INIMIIAIOJUI-CYUI UONL[IMOS eT] onb ja SULA INIMATAIOJUT | 
Op [[BAR] Op ddILYO LJ IS sJopour Jed d9\sISse INIMIIAIOAUT ,P 
1Q]J2,] Op anuj ey red souuop Anajea ve] anb a]qiej snyjd ysa 
ffat “Adq uejd un mod ‘anb tssne suoquow snou ‘uoTses 
a99 sueq (7961) Ysry Jed jUsWamMoa\Ue sdUUOp aapour 
qed agysisse uoneoynsnl ey ajajduioo inb ffa1 ap saningui 
Sa UONRIDICIOIUT OUN,P BES |] “INIMOIAIOIUI,P dII]IQeueA 
RF] ep 30 oseuUoTHUY,p oIqeueA ke] op SIoy eB] kL 
3}du109 jus mb uojpHueyos ,p suudXoul PT ap 9]e}0} DOURLILA 
BR] Imua}qo suoaAnod snou ‘/fai red ajduns amoyeaye adeuuo]]y 
-uevyo9,| Inod uojjHueyse,.p suusAOUW v] Op doUeLIeA PY JULI 
-VJNUI ud ‘oUOG ‘syoUSsOWTOY 3999 ap a}dut09 sed judT} ou 
Inb 3]909 UCU sJa=poUr UN sNOs d]]99 B MIMIIAIUT SUIOUI 
un Jed sarf[iandal SUOT}RAIASGO Sap 9}19UaSOW0Y, | 9p a}duI09 
yueUd} g[9aU09 s]9powl UN snos UOTTHURYDE,p suUdXOUT 
P] Op 9OULLIVA PI Op ORI ne [esa yuouIa{duIs ysa__/fa7 ‘sayes9 
INIMIIAIOU!.P [IVA op sasieyo sap mod ‘anb suoyuoUl 
SNON “ff21 INIMOIIAIDUI.P JOJJ9,] Op sjapow Jed agysisse 
uonejidiajyut aun suOSssTUMOJ snou jo saeneds soddeis 
sues qdqq ued un suolepisuod snou ‘7 uoyses eB Y 
‘orotnored ajanbua 
aun Inod UOTeUII}sSd UOS INS UOU Jd INDMITAIOUT.P SITIQVLUeA 
R] op uoT}eSId19}UI | INs 39 SUOIUTFAP SIT INS 191 SUOIUIIUOD 
snou snou ‘jIp yuaWIaQNY ‘ajgnbua,p sesuuop sap asAyeur | 
e.nb joynjd sajygnbue sap uoyeoyriueld ey e solqeordde 
NO} JURAR JUOS JpOI"e | Sup SayUasaid sye][NSOI Sa] “INOMOIA 
-IQUI.P dTIQeLIeEA ke] aIINpol op uye sosud ano JUdAIOp 
Inb samnsaul say JOUTULID}Op & SPIe INIMIIAIOJUI.P IOUBLILA 
R] Op IQely UONIUTap su, ‘asepUOS OP SOLIRUIS SIOAIP 
SNOS INIMIIAIO}UI.p soULLIvA Pv] ap sadtidoidde suonlusp 
Sap suouuOp snou ‘aoe JUasoid 3] SUBC] “INIMOIAIOJUL | B 
a9g}00]Je [IA] op ad1ey9 ey] op onb Isute ‘axa[dwio9 aseuuo| 
-[queyse,p uejd np oimeu ev] op puodap moMoIAIojUI.p 
oueLIeA ke] Op uUonIUap ke] aonb suoArlasqo snou ‘p ke 
Z SUOTIIOIS XNY “SINIMIIAIOJUI SAP SaTVSOUT [IBABI] Op Sadreyo 
d0Ae Fqq ueyd un,p seo oJ suep SUIOUT ‘SINIMOTAIOJUT So] Jed 
QSNVD J]P}O} DOULLIVA kB] IP JUDUIdSSIOINOB UN UTWOD 9g}oId 


-IajuI ay sed ynad ou “d({[ — “u) +] prepurys s[nuUoj 
31390 onb suousinos snou ‘z uonsas e] WY “d(T — “u) +1 
QINULIOJ eT adIp-k-]S9,9 ‘sIMaMaTAIOQJUL «sap suUs<oU 


yieaey op oasieyo ke] Jed a90vjduiol jso SMOMOIAIOJUT 


INAMIIAIBUI,P S}IIGeUeA e] 9p UONeJesJduaju!,| ap Jo UOIIUYSpP BE] aq : WIYe™ Ja Ja]GeH Z6 


“npo ‘pun Asains@)uryeyd : JOwUNoD “stup}-syeyq “Ye 23a]]oD 


‘purjArey] np syssoatuy) “UIYye] eyWeY ‘3Jo'stso3@)s9]|qQedpawysais : JaLuNoD “ousewla|y “WroyuURW! 71089 ‘SS IZ ZI ‘d’O “SISAD “19D peyyse1g °] 


Sop suNUIWIOD ]IeAe op adIeYyO ke] NO s[NULIOJ 9)]90 
ap ajdurts uoneoyIpoul oun yuouruonbay juast[yN saygnbue 
SQ] INS dYyoIOYyII ev] Op saysiyeloods soy ‘SINOMOTAIOJUT 
Sap sojesoul ]IeAey op sasreyd daAe saxajdulod sajygnb 
-Ud SOP Sed J] SUR “SINIMOIIAIOUI Sop IUNUTWOD [RAR] 
ap asieyo ey so “'u no “d(t — ™“u) + | sed juowayduns 
s9uUOp jsd uOTTHUeYIE,p ouUdAOU eT INOd IMdMOIAIOUI,.pP 
DOULLIBA P] ‘SINIMOTAIOIU! SOP SayTLVSd [IeAR Op SAdIeYO IAB 
Adq ued un mog ‘sInamatAioqut soy ed yUoWIonbrun sasneo 
Q]e}0} DOURLIVA PT OP JUOUIASsIOIONe.] JUR}9 SWIWIOD IUTJSpP 
JUSUIITeIDNUNS }SO IMIMOIAINUI.p ‘VoUPUVA NO ‘Jaqjo,TJ 
‘sjuedrorpred 
sked Qg][ mod IMOMOIAIOUI.P [IeAeR op sasdieyo sop 
9 INIMIIAIOUI-CUI SUOTLISUOD Sop sayoRISNOU eB SazI0q 
Sop unpoid yuo (f00Z) IPJaasooyT 39 suaddiptyg ‘osuusadoma 
g[e1o0s ajonbuy | ap seo a] sued “(986[ AAPTISeJ 19 SOAOIH 
S€Q6][ Jayony) sanbruoydsya} sajanbua say suep ajuanboy 
Sai] JSa (S¥O OL & C9 Op aJdwoxs Jed) JdAITO INIMOIIAIOUT,P 
auUdAOU ]IeAR) op asseyo oun,nb JoUsI[NOS op JUdTAUOD 


II “(S007 JoMeNy 39 TTeUYyDS) 69°T = fa! suoAR snou 
‘ol = “ue 100 = “d Is ‘ajdwioxe Jeg “S9A9T9 INOMATA 
-IQUI,| Op [eae op suUdAOW aBIvYO ouN,p osned ke JUdUI 
-g[ duns Jueyodut ang peLMod MaMdIAIIUI ,P JJJO,] “OPqQrej 
}SO INOMOIAINUI-eNUI UOnRaUOD eT IS swgUI ‘anb Jasued 


qul 


ve ouuop ™d({ — “) + | INOMOTAINUT.P JOYJo,] Op prepurys 

QINULIOF ey ‘stnayosayo smarsnyd susI~Nos jUo,] SUIWIOD 
‘(SQOZ) JINATY 39 TJOUYOS 0A ‘uOTsonb ke] ap 

atpuoyoidde snjd uorssnosip oun nog “SddAIasqO JUOUISTeI 


oddeis ° NOMOIAIOUI-RUL UONRIIUOS { SINIMOIAIOUI Sop [ICAI] OP sSadIeyo { INAMOIAIOUI.P IA 


yuos [9 B somoaodns sinaeA sop “(686]) SOAOID uUOTIS 
(1961 peysiog 19 ZIMmpP ‘uasuRH S9OC6T ABID * P96T 
ISAT © €L6T Jomeay © 786] JoyIng 39 suT[JoD) sojduns 
sayjamoery suonsonb say onb IMdMaIAIOVUI-eUL UONR[ILI0N 
RB] & SaTqisuas snyd yue}a DUTIOD SadIAPISUOD JUOS soxa{dus0S 
sayjomory suonsonb say 3d sopmje soy Ins suonsonb 
soy ““d op moyea eT IMs doUaproUT ouN MOAR ynod ajgnb 
-ud,p suonsonb sap aimjeu ey onb nuajnos yuo simmoyooyo 
sax) ‘(Ssamne.p 19 ‘[QNZ JOWD 19 GooH ‘ O86] AAvTIseP 19 
SOAOID £ €QGH] JOAON £ VSG] SP 19 UOsueH {£ 9G6] ABIH 
‘7961 Ysty) soyeneds soddeid sues sonbruoyds]a} sayonbus 
sop o}xo]U09 oJ Suep jUoWaTedIouLId ‘sMoyoIoyO xnoIq 
-ulou op Jed aaIpme 939 & UONRTaLI0N op adj 99 op dour} 
-iodur,J “'d INIMITAIOIUI-P.HUI UOYL[ALIOD op JUdIOIJJZo09 
np udXour ne samMsoul 939 & INOMOIAIQUI-VNUI gIDNUN3 
-OWOY,,| ‘JUSWIITOUUONIPeLL “INIVUITYSS UN,P d]e}0} DOULLIVA 
e] aJUaWISNe ‘UoNbssuod Jed a UOTTHURYDS,| Op dATDOTJO 
aye) ev] yowasyeiouss yinpsr mb say19uasowoy oun dajonb 
-U9,p sadsuUOp so] sUBp dIINpoNUT jnod MaMOIIAIOUT,T 
‘adepuos sed sqyinpoid sanbysnyeis soy suep INOMIIAIOVUT pp 
SJaJJa So] Ins Jonyur yuoAned mMaMoIAIOVUI.| op sonbysu 
-oeIed say 39 aMeuuoNsonb np uondaouos ke] op oyyenb 
eB] aonb spa} sinajoey saq] ‘sajgnbua say Ins oyoIoYydaI ey op 
saystqeroods xnaiquiou op sed pre} snjd 939 &,] 19 676 Ud DONY 
red nuuosal 939 efap year We} 9D ‘sajonbua say suep ammsour 
ap sindaua.p aueyodum somos uN }Sd MIMOIIAIOIUTT 


uoHoNpoyUy, «*T 


‘oyesoul uoNeiapuod ‘ ayeneds 
: $9]9 SJO/| 


"3[210} DOURLIVA P| IIINP9I 9p SUDAOU S]UDIAJJIP sIpusidwiod ke apre uONIsodurosap 
oN99 ‘“SINOMOIAIOWUI xneV jo sayeneds soddeis xne ‘uoyelgpuod ev ke snp sjajja ua jeqo]s jaJJa,, op uontsoduiossp 
aun issne suosodoid snoN “INSMAIAIDIUI.P DIIGeUeA eR] Op sJONUOD ne s[qeuUOsIeI snjd jospnq uN JQ}09}Je,p OUOp 
JOULNd Jd INIMIIAIOJUL.P S}oJJO Sop 939ex9 snjd UONENyeAD oUN UINO} s[NULIOJ 9NON ‘soyeneds sodde13 sap ja uONda]Is op 
sayeSoul soy[iqeqoid sop a}dwi09 Jud) Inb INMITAIOUI_P [IqeLeA ev] op saLidoidde s[nw0} auN suoUa}goO snou ‘axayduI09 
sseUuUO][HULYdE,p uR[d UN_p 3}x9]U09 2] SUC ‘d][oY499 opueIS e sa}gnbua so] suep sajusNbey soy jos ajeSauI UOIeIgpuod 
ev] onb sure sojeneds soddeid so] ‘slojayno] ‘sojesa JUOS sINSMAIAIOJUL Sop [IAP ap sadieyo say anb 39 uoT[MUeY9,| 
suep sojeneds soddeis op sed ajs1x9_u [1 puenb (4 gq) sojede sayyiqeqoid doar adeuUOT[NULYIE,p sapoyjoul say Inod snuuos 
USI I[NULIOJ 9399 ap s[gpour Jed a9g\sIsse UONROyNSsNf sun suOUUOPp snou ‘a[oIjIe JUasaId 3] SUR] “INAMAIAIOJUL,P [IVALI} Op 
sosieyo sop ojduns suusAoul eB] Jo INIMOIAIOIUI-EUI UONLIIU0 ev] ‘JUoUIDANOOdsaI ‘uos “uw yo ™d no “d({ — Mu) +] 
= jul ffap = ffat xed a9uuOp sd “INIMOIAIOIUI.P joyja sgjodde jUSANOS ‘INIMOIAIOJUI,P I[IQeUeA ek] op dINsOU 
ap onbisse[o a[NULOJ BT “INOMIIAIOUI _P S[IQeUeA k| DINpol op 19Aessa nod sagKojdurs jUOS sINAMAIAIDJUI Sap UOTEIJL | 
B 9 DOUBLIDdXd | B ‘[IBART] Op IBIeYO ke] & ‘UOTEULIOJ e] & Nb Isure ‘suoNsanb sap uoIe[NULOJ e] B Jd JULIO} Ne sodT] saIZo}eNs 
S9SIDAI ‘adepuos Jed soyinpoid sonbysyeis sop apIqeueA ke] ap ajuepOduT aURSOdwOd dUN jSd INIMIIAIOJUL, P I[IQELIBA BT 


SUINSIY 


, AYR] eyed yo Aa[qey pataysais 


dxo[dwi0d osvuUOT[VULYII,p uvyd un 1n0od 
ANIMITAIIJUL.P 9TPIQUIAIVA B] IP UOIVLI9AGIIUL,| OP 39 UOTIUTIpP &] 9G 


L6 


enBojeyeo ne x-100-Z) .N ‘epeued onbiysyeys 
9OL-16 ‘dd ‘} .N ‘SE ‘IOA 
6002 uINn! ‘ayanbua,p sanbiuyoa | 


‘Od “uoysuryseM ‘(01°70 UOISIO A) 
fenuey] souss3Joy WWIYV ZI-X (ZO0Z) neeing snsuag $7) 


‘Be]I9 A-Josuuds 
> YIOX MAN ‘AUD YOR MON “Saidag aut] 4of aouasafuy DI1jS01S 
fo Mioayy ayojdudsy (QO0Z) “A ‘emezryey 3 “PW ‘Iyonsiuey 


‘ov I-VHl 
‘uoneIDOssy [eosyRIG uURdLIOUrY ‘UOI1j2ag SoIISIIDIG DWDUOUu0IT 
pup ssouisng ays fo ssuipaaz01g ‘jpd's¢g66u/sioded/sy/qnd 
/AOZ3’snsusa MMM Ne Issny jUoWYsnfpe Jo Zuljapoul Jaye syayyo 
Aep 3urpey jo souasoid ay} ayeoIpul 0} WWIRIV-ZI-X Aq pesn 
sonsouserp wingossds a UO (6661) ‘AC ‘Aolpuly 32 “py ‘drjnog 


d||enpises a}ljeuUOsies ej INOd anbujawWesed UOU }s9a} UP : UBJOH 3e AoujQoyy 


ondoyeje9 ne X-[00-ZI .N ‘epeueg onbysneis 


‘SSolg oTWapRroy 
:UOPUOT ‘saldag aul pun sisdjoup jp4j2adg ‘(1861) ‘W ‘Aepseuid 


“LvC-177 “PUeIIOH YHON 
: Weploysury “(yeleuYsLy “_ 30 Josul[ug “q “spq) ‘/] sousuvig 
fo yooqpupyy “uoryeurysa yeyoods daissaiso1oyny “(€86]) “A ‘Uozeg 


ES T*CS | 
°Q/ SUOIDIDOSSP JDIISNDIG uDIUaUp ay] {O ]DUANOL *S21L13S BWI Ul 


spouod ZUIUTWD}ap JO} poyyoW Y “(€861) W ‘ouersed 39 “H ‘UOIMAN, 


‘987-1 PZ ‘TE ‘valdjauouoog “sainpos0id 
juowysnipe yjeuoseas jo sisAjeue jeyoods (p96) “We ‘AOLION 


‘sa. opq’ MMAM//:dyy ‘uid fo yung ‘juaujivdaq 


YOADASPY ‘“pOOT [leaky op juoWIND0q ‘JenuRY sOUdIIJOY 
PIstAsy MSL Weisolg (7007) “DH ‘oy[esodeg y9 “Y ‘[Teaesep| 


06 


anZo]eie9 ne K-[00-Z1 ,N ‘epeued onbysneis 


‘epeued onbysneis 

‘sanbidojouoIys souigs ud osAyeue jo SYOIOYOOI ke] OP UOISIAIG 

qeAe ap yuowno0g ‘wreisoid WWIYV-II-X oy) sulsn uaym 
Ayjeuosvas a[qeynuapt Joy say VY (8/61) ‘WW “AuOoy 39 “f ‘uRIpOT 

“ESI-ZEI ‘OL ‘sousunig fo sjouup ayy, ‘sassaooid 

Jeoul] JO uoneU}sa JojoWeIed ay} pue sassaooid Areuoneis 
JO} W910dy} yu] [enUI. Y “(Z7861) ‘W ‘Tyonsiuey io “x ‘eAosoy 


‘(uoIssnosIp 99k) LL 1-LZI “OI ‘SousuDIs 
a1uoUor_ pup ssauisng fo joudnor ‘weiso1d yuouNsn{pe [euoseas 
VIIUV-ZI-X 241 JO spoyjour pue somipiqedes MaN (3661) 
‘O'd ‘UaYD 9 OW “ONO “WM ‘Pd “O'd ‘Tesuow “Ad ‘Ae]puty 

‘g]]QuUOSIod UOYRITUNWIWOD (9007) ‘q ‘Ae[puly 

{VA ‘eupuexely : [WOU-dd] ‘vouniossy 
[DIYSIDIG ubIdaUp SsUuIpaar0{d 9YQZ ‘Seed jenoods Sutssasse 
Jo} soinseou Bunenyeaq (9007) “L ‘AOJ[AIW 39 °S “uelOPH “y ‘sueagq 


‘OTEI-STET “OI ‘sousuvigs 
fo sjouup ayy ‘sauias ow e jo siojowesed oy} JOJ ulewop 
Aouonbay ay} UO sioyeursa sarenbs seo] paysiom, (8861) *S “NUD 

‘AbQ-UdplOH : oosiouel{ 
ueg ‘d1oayy pup sisdjpup vjDG Saldag aut ‘({[861) “d ‘Josuly[ug 

‘O0€ 

-687 ‘LS “q Salas ‘“Aja120g poosinjg poxkoy ayy fo jousnor *Bunysa} 

ajdyjnur 0} yoeoidde jnpiamod pure jeoyovid y :o}e1 AIOAOOSIP 
as[ey oy} SuJonUOD (S661) “A “S¥qyooH 1 “A “uruefueg 

‘[[eH nud 

: Aasiog MAN ‘SHO poomasuq ‘soidoy pajoajag pun svap] 
Ispg :sousIg jooyoweyinN (LL61) “A “Wnsyod 2 “d ‘TexIg 

‘OTE-167 “7 “SAUSIIDIG D1M0U0DT 

pup ssauisng fo ]DuANO¢ “SaLIaS dW dIWOUODa Jo jUsUySN{pe 
[eUOSeaS dU} YUM PSA[OAUT Sanss] ‘(PR6]) “S “OWN 19 “M ‘T9 


‘npaang Snsuay ‘$A ‘pI-ZO0Z SY ‘ON 14oday youwasay 
dus ‘sunsessio, pue juoujsn{py yeuoseag 0} uorjeoyddy 


Noy], pue soueg sw] [euosesg IOJ sjpspopyY VWIWIIV MON 
(L007) ‘Cl “UIpRY 19 ‘STI “LL ‘Aoapqo “q ‘Agypury “f ‘uoisy 


arydeasonqrg 


YAM AH satdyynus 
$}S9} op sinpsooid ey Jajadde yuanbgsuos zed suoAnod snou 
32 onbyojduiAse souepuadaput | suouajqo snou ‘yu1ofstp ysa 
xneAou op omed ayjanb oyodumtu mod yioddns 9] 1s ‘ouog 


iP) BOG AE Ole 
Cisthe 


senus ff e] ee 4 onbyo}duiXse 

JOURLIRAOD Op dOLNeU eT ‘YNs [I ouTWIOD xneAoU p 

® xnop op Jossed mod | sWig109U} J] JosI[e19UaS JUSUITIOR] 

suoanod snou ‘slo; y “7 UOT}Ses eR] Op sasayjodAy say uNoeYyo 

yuojsnes nb ‘p “ ‘7 {| = 1 mod ‘p xneAou ap oaruy 

aLI9s 9uN SUOAR snou onb duop suosoddng ‘saydyinur soid ap 
UONN9}9P IP 189] op OLIRUDOS Np sUOINdSIP snou ‘ayInsuq 

"4 990U0U9 

SOURLIVA 9p SOLLUI ke] SUOSINpap snouU ‘suUOSTeUTqUIOD 
SasIdAIp suep UN jo OINZ B xneda 4 Jo x yURUaId UW 


68 


“4 
(0 beldnd o's 
c 


CRBs) Mees eaty 


(Og ()¥ == — 7+ 


wy) ul 


OOoK an ag: GaSe 
iG 


2 0 = (Y-)D (YO ‘eoueptag 23no} oq] 


(a8 Caen 
coy fa + eee 


red Typ isa D neAou 2] No “(7/61 
uinsyoq 12 [axoIg) AysINIg ep swWolosy) a] jJUeSITWN Us 


= (WO 


eac) £02 + C900)" = oN = 


(ais earns: Ju 
G)%-m*  We-we J 


‘Jaer & ‘x yno} mod ‘ouog “((886T) 
ny) wp ¢€ SUIDIODY} I] IIOA) somedu suoToUOJ say] amMyouT 
Inod jUsIAUOD ]I SUMO 9sIfeI9Uas ‘(CQ00Z) eMeZTyey 
yo 1yonstuel op [TE su, of suonbiydde 39 prom 
-I9WIkID op onbruyos} ey suosijyn snon -(7)%9 19 (7)"6 
ap Aulofucs souas19AUOd eI SUOIOPIsSUOD ‘UdWIDWIOTXNIG 
‘py op sored ey e g dace JUOUUON sjyey[NsoI souWIoW 
Sq] ‘UOWMWOpIAg “(wZ IMaOeJ un doae oWWeIsOpousd 
9g] Issne yuIyop (9861) ny) UUReWorY op souruos say red 
QeISHUI UONUOJ eT op UONeWMTxOIdde | a1ey ][I,nb juRj0U 
ud “(8g61) NIYD op | aAzleJoIOD np jUoWIA}DaNp s[Noosp 
yeynsor 99 “([Z7/g + 4 ‘7/g — M1] onb Joy) ayfearoyun 
un suep nunuos jso Pp neAou gf anbsing (,/) "Oz 
a7 (eD "9 onb suossyqnig ‘1s0usUTWIOD Nog ‘aanaig 


nt 9 age A eh 


Coes: 


JUOS SAAdJUA SA] JUOP ZT X JT UOISUAUIP ap aD1AJDUL BUN Jsa 
A 12 ‘(Q ‘Q) [nu anajaaa aj ajouap (~) ‘10 “0 < u puonb 


| zie e ; 
()% - 1a) 


crn eo, | 


+ 


CASON = 


oT 


re - rea 

SAOIY “|Z UOYaS DI ap (At) 2 (2) 
SuOYIpUuod Saj juvsinf{syws g Ja p xnvdou saj flog ‘aatfisaa 
182 (8861) YD ap (8) 1 aseysoddy anb ja ‘ aarfisaa jsa 
(LH) no (g) uoyipuoo vj anb ‘ juassipavdsip {'x} ap aspso 
auaigonb ap sjupjnund say anb suosoddng | awasoay 


600Z ulnf ‘ayanbua.p sanbiuyoo] 


"S89 XNOP $99 dIANOO Inb |e19UN8 dUIAIONy} 
3] SuoJUsssId sNnON' ‘PY NeAOU SUIQWI 2] DdAR d}IXdAUOD 
ap 30 ajuod op ainsour oun.p m3e.s jnad I ‘ajdwiexa sed 
‘(7)’ samsaur ap aired ayno} ap ayutofuos souasIaAu0d 
oun 4so [ediouud yeynsal a7 (LH) wWeysnes syusutowr aynenb 
DAB SIIVOUT] [esned snssaodoid un ‘aydurexa sed { ayoLNs 1s9,U 
SUOIIPUOD S2d 9p sUNONY “_D suUep jsa afeNoods 9yIsuep 
eR] JUOp UaIssnes snssadoid un sed ‘ajduraxa sed ‘ayreysnes 
}so Inb ‘9Ad[9 SIPIO.P sJUR[NUIND SIOAIP INS 9yITIQeUTUOS 
ap UONIpUod sUN Iso (Yg6I) NIYD ep (g) | esamoddy, 7 
‘SOQIJLIZA JUOS (9¢-¢¢ saved ‘QQQZ) BMLZIYEY 39 tyonstue ] 
2P (9H) & (1H) SuonIpUOd sa] Jo Sea] UONRIUAsaidal ouN B 
snssad01d 9] onb souous (7861) ryonsiuey 3a eAOSOY & onp 
(LH) UonTpuos eT “99IfUI9A Iso (Cg aded “QQ0Z) eEMEZEEY 
39 1yonstue] op ([q) UONTpUOdD FJ Jo aMIeUUOT]E}s JUDWID}OLNS 
yso snssadoid aj aonb aouous ‘([g6]) Josut]Ug eB onp ‘(q) 
uOIpUuod BT ‘(T'['E UONdAS “QOOZ) EMEZTABY 30 TONsrue], 
ap suomdsur snou snou ‘anbyojdwAse suocey) ey imod 


0 Lv‘0 £50 81°0 Lr'0 

LIU/LI LI/S1 LIU/E€l LI/9I EUV PEEVE LT 
g g g g g 060 1€0 
g g g g g COLE 0 
g g g g g 190 =£9'0 
g SveTcl veTl g veTl Sto S00 
g tcl vl tcl vel rl'0 =7%r0 
g g g g g 850 810 
g g g g g 160 670 
g g g g g 9L°0 S70 
g g g g g cy 0 WLIO 
g g g g I Lv‘0 SI0 
g g g g g l€1 39°0 
g g g g g 6r0 070 
g g g g g 950 £70 
g g g g g Or'0 910 
g g g g g 650 ¥9'0 
g g g g g 760 £50 
Gg g g g g 8sI 96° 

HL anbyiengd HL anbyuiend 

AS 010/010 WAMA-H SOO/OLONAMAH 8W LI 


sagsT[BuUOosiesap seguu0g 


ondo[rje9 ne X-[00-Z] ,N ‘epeuea onbysyeis 


S99UUOP S23] INS sasInbal yUOS sa]qiey suOTIPUOD sauTR}IAD 
‘sonbnoyduisse sonbyuo sinayeA sap iImuajqo mod jsaq 
op swisipered 9] suep sanbrjdde ane sioje yuaanod syeq[nse1 
SIC) ‘SIIX9AUOD PI Op 39 Quad eT ap / sh sanbysye}s somsour 
sg] Inod sonbyo\dusAse saynuo} sa] 191 sUOSsT]quIa SNON 


dxouuy 


“STE/ASN/VSV 1 
ap sysIsYydoI sp asmoq sun Jed sour 939 JUO UROH 


ap xneARy say ‘neomNg snsuad “S'- NP sayjao UdWIS.IO} 
sed juajuassidal au ja smoayne sap saad yuos sasurudxa 
suoluldo sa] ‘sInod Ud XNeART] SAP UOISSNdsIp k] IISLIOARJ 
op 39 aURINOD SYOIOYIOI vk] Op jULINOD Ne sagssaI9]Ul 
sorted sa] Id} ap onA Ud YSNYFIP so opoIpe Juasoid a] 


SJUIWIIIIAIUIIY 
9L‘7 90'r p6€ QL‘€ OLE uaXOul 21QUION 
« $1D0.109 >> 
LULI LILI LULI LULI LULI SJEINSHI Op [BIOL 
€7 & €7 €Z €7 IOVAWILX 
€7 Sbel Sbel pel rel 1OVAXALX 
SbTI SPeTl SpeTl SheTl SPETl 1ODYA TINA 
SPETl SPeTl SPeTl SreTl SPeTl ZOOLLUTS 
71 Stel Srel Srel Sel COUOLUTS 
p£Z rel 7a rel v7 IODANWUd 
pe7 peTl peTl veZ1 peZ1 COUANWUd 
SrETI SPeTl SPeTl SpeTl SrETI ZOAANWUd 
€Z Srel Shel Srel Srel 1OOLNWUd 
€7I SPE7 Svez ShET ShETz IODINNYd 
ZI peZl €7I ZI €7I IOSONWad 
€7I SPeTl Spec SPeTI SPezl COOONWAd 
€ZI ShEz Spez Spez SheZ ZODONWUd 
€Z Stel Srel Sbel Srel [OO.LNIUd 
ZI SPeTI Srel SPeTI Srel 10NAdIdd 
Z €ZI €TI I I ZO0dWNVIdd 
v7 pez reTl peZ1 ETI 10dWVIdd 
ANDV.LAUA 
HL anbguienQ = HL = anbyiend -AGNVUD 
AS OL‘0/0l'0 MAMA-H_ ~—-S0°0/010 WAM A-H aLl9g 


S3[BUIGLIO saguUU0g 


auseja1g-spuely “qQOQO — seauuop sap sasAjeuy 


(Zp UOIZIIS LT JIOA) QI‘O = Q zB 2}UAd EU] JUIWIEXA 39 QI‘Q= VD JI CN‘QO= VD 
& [eqols | adAj ap anaisa,p xnv} af 19[01}U09 anod YAAMA-H 2poyjzow ey ans aindde,s sajdyjnui said ap dNsouseIP 31}0N “SIA 3° LW 
‘AS SdQSOUsEIp Sa] daAv Safdyinu ssid ap dysoUuseIpP 2.130U JUKALdWIOD YQDO,I IP Senbruuezq satigs {] 4nod saguuop sap saskjeuy 


a|jenpise ayljeuUOsies e] JNod enbujawesed uOU }s9} UN : UR|OH }e AoujQow 


OT neafqey 


88 


andojeje9 ne X-[00-ZI1 ,N ‘epeued onbysnes 


0 18°0 €L'0 9€°0 9£°0 Sve 9p r9'P b9'P p9'P uoAOU D1QUION 
« $}90.109 »> 
IT/TI 11/8 11/8 11/6 11/6 GS Ab IT/Tl T/T it Ae I/I1 LNA Se ee 
g g g g g rs'0 810 SE Svcl SbCl Syl Syl IOVAWILX 
g g g g g rr0 10 €7 Sel Srel Svel Srel IOVAXHLX 
G g g g g 9€0 910 Srecl Srecl SreTl Svecl Srecl COOLLYTS 
g G g g g LEO. 2c 0 tcl Sree Svel Svec Svel IOYOLATS 
g SvTl Svcl g g re0 70 Srecl Srecl Srecl Srecl Svecl lODANWad 
g vecl tTl tcl tT 670 910 Srecl Srecl Srecl Srecl Srecl IOOLNWud 
g g g g g 9770 ~=—«IT0 Srecl Srecl SPCTl SveTcl Svecl IODINWad 
g g g g g 8s0 sto vec crecl Svecl Spel Svecl IOSONWad 
g I I I I 80 STO vec Srecl Srecl Svecl Srecl IODOONWad 
g g G g g rr'0 =r '0 Sec SVEC SveT SvET Svec IOOLONUd 
g g Gg g g 67 OM 210 tT creel Svecl Svecl Srecl l0OOVAV ad 
HL anbuiengd HL anbyiengd HL anbyiend HL anbyiengd FAONVU 
AS 010/010 WAMA-H SOO/OLOWAMA-H  =68INN OLIN AS 010/010 WAMA-H $0°0/010 WAMA-H a9S 
sagsTjeuUOsIesap saguUOg SI[VULSLIO ssQUUOG 


gue. y “AdQO — seguuop sap sasAjeuy 


(Zp U01}99S BT AIOA) O[‘O = Q & dQUAd Ee] OUIWIeXA 39 QI‘Q= VD JIS0‘Q= D0 
B [eqoys | dA} ap anai12,p xnv} 3] 19[0.143u09 nod YAAAA-H epoyjoui ej ans sindde,s sajdyjnw ssid ap IVsouseIp 3430N “SIA 39 LIA 
‘AS SoysouseIp say 30Aze sojdijjnui soid ap dsOUSeIp a4j0U JUvIedWIOD FQDO,| IP Sestesuvsy satsigs [] 1nod saguuop sap sasAjeuy 


6 NesyqeL 
0 03°0 L3'0 €1°0 070 CLS 09°F Or'r 09'r Or'r uoXoul a1QWON 
« $}90109 » 
SI/SI ST/II SI/Ol SI/€l SI/OL  SUST SST SI/ST S/S S/S S/S SI/SI SIEYSST OP HOD 
g g g g g 7L'0 =—(Ov'0 €7 SvLT SHCT SveT SveT IOVAWILX 
g Cl Cl g g Ls0 =—«1€0 €C SVEc Sel SULT Svel IOVAXHLX 
g g g g g IO S00 SPrEzI Srecl Svecl Srecl Svecl COOLLYTS 
g g g g g 710) 3=6S00 «= SPrETI Srecl Svecl Srecl Srecl IOOLLYTS 
g Spel SvTl I I O£:0 LIO FSrEez Svecl Svecl Sretcl Svecl TODANWTS 
g ve vc g ve Ir0 070 eC SPELT Svel SvEc Svel TOOLNWTS 
g g g g g Se WA) €C Svecl SveTl Srecl Svecl CODINWIS 
HL anbyiengd HL anbyiend HL anbyiend HL anbyiengd ound 
AS 010/010 WAMA-H SOO/OTO WAMA-H «68S OLW AS 010/010 WAMA-H $0°0/01°0 WAMA-H 89S 
sagst]euuosiesgp saguu0g Sd]BUISLIO saQUUOg 


oiny “AdOO — seguuop sap sasAjeuy 


(Zp WOIZIaS Ef AI0A) OL‘O = Q & ajuad ev] aulMeXa j9 Q[‘Q= VD J9SN'Q= 0k 

[eqojs | ad4} ap snai19,p xnv} a] 19]013U09 anod YAAMA-H epoyjowi e] Ans aindde,s sajdyinw sod ap snsouseIp 34j0N “QI 39 LW ‘SAS 
SdISOUSEIP Saf JaAv Saydyjnwi ssid ap JNsOUseIp a.1j0U JULIedWIOD FQDO,| IP 0.1Na auoz eT ANS Saligs GS] nod saguuop sap sasAjeuy 
(a}Ins) g neafqe 


248 6002 un! ‘ajanbus,p senbiuyoa] 


g g g 
g g g 
g g I 
g g g 
g g g 
g vecl vel 
g g g 
g g g 
HL anbyiengd 
AS 010/010 WAMA-H 


g 
g 
g 
g 
g 
I 


g 
g 


HL 


g LS‘0 
g €7'0 
g rail 
Gg L70 
g Lr'0 
I 7€0 
g €7'0 
g pro 
anbyiend 


S00/0I'0 HAMA-H ~~ 8IN 
sagsTeuUOsiesap saguUu0g 


oiny “AdOO — saguuop sap saskjeuy 


1¢0 
£10 
am) 
S10 
770 
710 
Oro 
am) 


LW 


€Cl 

Sec 
SvEC 
SvET 
veel 
Svecl 
SPeTl 
Svecl 


AS 


SETI 
| 
SVec 
Creel 
Creel 
Spl 
Spel 
Srel 
HL 


Svecl 
vecl 
SvET 

Svecl 
veel 
Svc 

Svecl 

Sve 
anbyiend) 


O1‘0/0T0 WAMA-H 
S2[VUISLIO sagUUOG 


onsojeyed ne X-[00-Z1 .N ‘epeued onbysneis 


Stl 
Srecl 
Svec 
SPecl 
SVecl 
Svcl 
Srecl 
Svel 
HL 


Svecl 
veel 
SVET 

SveTcl 
veel 
SvCl 

SPecl 

Sve 
anbyiengd 


$0°0/010 WAMA-H 


CONONWTS 
[ODANWUd 
IOOLNWad 
IODINN ad 
IOSONWUd 
CODONWAd 
IOOLNIad 
IOOLNOUd 


ouna 
al19g 


rl‘0 00°I vil 


CC7/0T cC/91 cC/F 1 


g g g 
g g g 
g SET SVET 
sé SV SV 
g SveTl Svcl 
iS g I 
g €Cl vel 
g tcl vel 
g g g 
g g g 
g g g 
g g ve7 
g g g 
g creel Srecl 
g g g 
g g g 
g g g 
g g g 
g g g 
g g g 
g g g 
g g g 
HL anbyiend 
AS Or‘0/010 WAMA-H 


CC/TC 


Qe 


2 QTR Qa ene Qe en sgneaeasveeexee eee e se 


HL 


TOT = TLC 
560 
LL‘0 
650 
€s‘0 
€5‘0 
8r‘0 
z9'0 
z9'0 
670 
S70 
1S‘0 
99°0 
590 
1S‘0 
g €9'0 
g ZL‘0 
g or'0 
g 9r'0 
g 

g 

g 


DB) ye Sy RSV 


2 QQ RR SRS 


a 
= 


9F'0 

8r'0 

6L‘0 

g €c0 
anbyiend 


S0°0/0I'0 HAMA-H_ ~—8IN 
sagsTjeuUostesep saguu0g 


(Zp UOTZIIS LT AIOA) QT‘ = Q ¥& ajuUad Ee] JUTWIEXa 39 QI‘Q= VD J9SN‘Q= De 
[eqo[s | 9d4j ap anaiia.p xnv} 3] 13]0.13U09 anod WAM A-H 2eoyjsw ey] ans aindde,s sajdyjnw sid ap sysouseIp 3430N “QW 39 LIA SAS 
SITJSOUSLIP Saf JaA¥ Sajdy[Nw sdid ap INsOUseIP 31}0U JUKIedWIOD FQDO,| IP OAN2 9u0Z eB] ANS Sati9s S[ anod saguuop sap sask[euy 
§ neaqeL 


COCE 
1€°0 
870 
61°0 
v7'0 
0c'0 
€7'0 
070 
070 
€1'0 
Z1°0 
61'0 
070 
170 
z€'0 
70 
LE0 
70 
670 
970 
870 
val) 
L1‘0 


LW 


ETE 


COEC 
6 
26 

€cl 
cl 
cl 
vl 
xa | 
tcl 

Srecl 

Svecl 

vecl 
cc 

(36\! 
Cl 
ec 
tT 
x6 

SVeC 

Svecl 

Srecl 

Sec 

Svecl 


AS 


00'r 


CC/CT 
Svecl 
veel 
Svcl 
vec 
Sve 
tC 
SVEC 
SPEC 
creel 
SveTl 
cvel 
Sve 
SET 
Svecl 
SVec 
SVec 
Secl 
cecl 
vel 
Srecl 
vecl 
Srecl 
HL 


Svc 
SVEC 
Svecl 
SVEC 
Svel 
Svcl 
Sel 
vel 
Svecl 
vecl 
Svecl 


Sa[VUISLIO sagUU0G 


auseWlayTy “AdDO — seguuop sap sesk[euy 


S6€ 


CC/CC 
vel 
veel 
SvTl 
vec 
Sve 
€C 
SVET 
SULT 
Svecl 
SveTcl 
Svel 
Sve 
SVET 
Srecl 
SVEC 
SvET 
Stl 
Sel 
vel 
Srecl 
veel 
Svecl 


anbyiengd) HL 
010/010 WAMA-H 


98°¢ 


CC/CT 
vec 
veel 
Sec 
vec 
Sve 
SC 
SveT 
SvLT 
SveTl 
Svecl 
Svel 
Svc 
SveT 
Srecl 
SveT 
Srel 
SvTl 
Sel 
vel 
Srecl 
vecl 
Svecl 
anbyiengd 


$0°0/010 WAMA-H 


udAOUW SIQUION 


« SJOOMIOD »» 


S}EYNSOI OP [EOL 


IOVAWILX 
IOVAXYLX 
1OOYLYNN 
LIASAINA 
VWASAINN 
1ODYA INN 
COOLHM TS 
IOOLHMTS 
COOLLYTS 
lOOLLYTS 
1OYOLATS 
IOOLNWIS 
LODINWTS 
LOXHNWIS 
LOWGNWTS 
IONONW'TS 
IOGONWTS 
IODANWad 
IOOLNWdad 
LODINWad 
IOSONWad 
CODONWad 


ANOVINATTV 


aLi9g 


(Zp U01Q99S ET AIOA) OT‘O = Q B djuad Ee] JUIWIEXA 32 OI‘Q= 0 32 G9‘0= VD 
& [eqoys | ad} ap andi1a,p xne} a] 12[01j3U09 Anod YAMA-H epoyjsui ey ans aindde,s sajdyjnwi ssid ap dysouseIp 34j)0N “SIN 39 LI 
‘AS SsOUsEIp Sat daA¥ safdyinui ssid ap sHsouseIp 94j0u uLALdWIOD YQDO.I 2P Sepuewoae satigs 77 1nod saguuop sap saskjeuy 
L neosqey 


a|Jenpises ey\jeuUOsIes e| Nod enbuye@weJed uOU }s9} UF) : UBJOH }@ As OW 


98 


anZojeye9 ne X-[00-ZI .N ‘epeued onbysnes 


0 0 0 0 0 EGS 8 LL’ Cv Lov uaAOUl AIQUION, 
« $}OQ.I09 »> 
OL/O£ O¢/O£ 0£/0€ O¢/0€ O¢/O£ O€/67 OL/OE OF/OE OF/0E O¢/0t 0£/67 OL/OE S}EY[NSDA Sop [BIQUDS [k}0 | 
a oe ee EO, Sar er eer as TOW Rae OT ae CT Ger imme jess, 
OL/Ol OL/Ol OI/Ol O1/0l OL/Ol OI/OI OI/OI O1/Ol  Ot/OI 01/01 Or/ol OL/OI $72I[NS91 Op [B30 
g g g g g ECO PLO Sec crecl Srecl Srecl Svecl 00Srr 90S 
g g g g g ceO FEO Sech Svecl S| Specl Srecl Ol yyy 90S 
g g g g g ceO 9T0 SECT Srecl Srecl Sretcl Srecl OOrbY 90S 
g g g g g 8r°0 =1€0 SPrETI veel veel vecl veel c1evy 90S 
g g g g g 810 7IO SPrezl SPreT vecl crecl yecl O0OEry 90S 
g g G g g LTO €10 SPEZI SHETI Srecl crecl Sel 00Crr 90S 
g g Gg G g SHOMMITO “SET Srecl Svecl Specl SPvEeTl Ot I vr 90S 
g g g g g Oro 70 SEI Srecl “SPETI Srecl SETI OO ry 90S 
g g g g g 970 TIO SrEZ SPrETI SPecl Srecl SPrecl 000rr 90S 
Gg g g g i) LHiOiCeO, SEL Srecl Srecl SPecl Srecl OXI 7raos 
HL anbyiend HL anbyiengd HL anbyiengd HL anbyiend) aLlag 
AS 010/010 HAMA-H $0°0/01'0 WAMA-H 8N LIN AS 010/010 WAMA-H $0°0/010 WAMA-H 
sagsT]BuUOosIesap soguU0g S3[BUISLIO saquU0G 
[[B19P Ip sa}JUIA Sa] Ns saLiag 
« $}99.109 »> 
OT/0I OL/Ol O1/Ol OI/0l OI/0I 01/6 OI/Ol OL/OI  Ot/ol O1/Ol 01/6 OL/ol Syeq[NSAI Op [BIOL 
g g g g g LFi0., 600 4 SI vecl vec) vecl vecl OvIOIx 
g g g g g rol 690 €2 SPrecl crecl Srecl Sel CCOEX 
g g g g g OL‘0 ~6£'0 SPEZI SHETI Srecl Srecl SPeTl OcOLXx 
g g g g g L60 950 & vel vel vel vel 00¢€00x 
g g g G g v60 LSO SPrEt SHETI Sel crecl SPecl €X 
g g i) g g LLORES OY SEL Srecl Svecl crecl | O90TIN 
g g g g g SORE S'O PSPET VSPETIT vecl vecl vecl OLOtW 
g vo) g Gg G $60 870 £7 Srecl crecl SPecl Srecl O00LW 
g g g g g 6S0 8€0 SEZ Srecl Svecl SPEcl Sel 06100W 
g g i) g g Shirt Urn Sol Srecl Svecl g Sel OTIOOW 
HL anbyrend HL anbyieng HL anbyiengd HL enbyrend) alas 
AS Oro/Ol'0NaAMA-H = SO'O/OTOUAMA-H =8N OLN OAS OOOO WAMA-H ~~ S0°0/01°0 WAMA-H 
sagstjeuuosiesap saguu0g S3[VULSLIO ssguU0g 
su0ye}.10dxa/Suoyne}IOdUIT Sa] INS SaLIIS 
« $}90.109 »» 
01/01 O1/OI O1/Ol OL/Ol OI/OI OI/OL OL/OI OL/Ol OT/OI 01/01 O1/Ol Or/Ol SJEYNSSI OP [230]; 
g Gg g Gg g 9500 GB vecl Sel vecl vel OLM 
g Gg g g g py O - 120° S571 crecl Srecl vecl vel wey M 
g G g g g tvo0 O70 = SZI Spel Srecl Srecl Srecl LSN 
g g g Gg g 66.0. .LL0. Sal Srecl SPrecl SPecl Srecl we4qiSi 
g g g g g LS0 670 SZI Srl Syl vel vel I0LS 
g g g g g LV'O420C.0 SSC Srtcl Srecl SPecl SPrecl we4its 
g g g g g LTOISSCO Feel Srecl Srecl Svecl SETI OLHN 
g g g g g COO 910  eEl Specl Srecl crecl creel wejtTHNn 
g g g g g 180 180 21 Srecl Srecl Srecl SPecl JOLMAN 
g g g g g Se0. °Eh0 <I Specl crecl Svecl Srecl Weyl MW 
HL anbyiend HL anbyiengd HL anbyiend HL anbyiend) alas 
AS 010/010 WAMA-H $0°0/01'0 WAMA-H 8N LW AS 010/010 WAMA-H $0°0/01'0 WAMA-H 


sagstjBuuosiesap saguu0g 


Sd] VUISLIO sasQUU0G 


UOHRILIQE] BT Ns SaLigg — saguuOp Sap sasAjeuy 


S8 


(Zp UII BT] AIOA) OTD = Q 
& ajuad ev] aUIWIEXa 32 OI‘Q= 0 39 SO‘O= DO & [eqolsS | 9dA} ap ANatia.p xne} af 12[013U09 Anod YAMA-H 2epoyjsu ey ans aindde,s 
sajdyjnw said ap IsouseIp 9430N “QI 19 LI SAS S2sOUseIp say daAz Sajdyynw sod ap dNsoUuseIp a4j0U JUBIVdWIOD ([IE}9p NE sazUIA 
SI] ANS QT 39 SUOIeIIOdXxa/suOnEIOdUIT say Ans QT ‘jUIWIadO] a] INS QI) Neaing snsuaD “S‘p NP satsas OE nod saguuop sap saskjeuy 


9g neajqe L 


6002 un! ‘ayanbua.p senbiuyoo | 


ango[eje9 nev X-[00-Z1 ,N ‘epeued anbysneis 


GEIS EC erSebwnr 16 S/E Siw he SEISE SE/SE 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g PET peTl g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 
g SpEZI SETI g g 
g g g g g 
g g g g g 
g g g g g 
g g g g g 


HL anbyiend HL anbyiengd 


010/010 HAMA-H_ ~—-S0°0/01‘0 WAMA-H 
sagstjeuuosiesap soguu0g 


> 
N 


SE/EE 
9¢'1 
99°0 
S30 
80 
LEO 
960 
S90 
el 
rb 
770 
z79°0 
aa) 
er'0 
Lv‘0 
sr'0 
50) 
80 
00'I 
LEO 
99°0 
Oro 
€r'0 
6£0 
s°0 
xa) 
ram) 
€L‘0 
ia) 
6L°0 
6r'0 
Lr‘0 
rb 
9r'0 


St/ve 
cel 
S70 
9r'0 
¢7'0 
170 
19°0 
Sr'0 
8L°0 
Lo0 
S70 
LEO 
L70 
970 
070 
970 
Le0 
95°0 
98°0 
070 
79°0 
€70 
£70 
al) 
0c'0 
S10 
80 
zr'0 
81°0 
sx) 
870 
L70 
870 
870 
070 
v7'0 


LW 


AS 


Stcl 
HL 


60°F 


tcl 
SveTl 
SVT 
Srecl 
Svecl 

SPCC 


anbyiend 
010/010 WAMA-H 


Sa] BUISLIO sagUU0G 


UOHRILIGE] I ANs saLigg — saguuop Sap sasAjeuy 


67 € 


SvETl 

SveTl 
Sec 
HL 


60'¢ 


SE/EC 
SPeTl 
Svecl 
SreTl 
SPecl 
Stcl 


Svecl 
Shel 
SVET 


anbyiengd 
$0°0/01°0 WAMA-H 


UdAOW BIQUION 
« $}]99.109 > 
$]2)}[NS91 IP [20], 
he Ww 
St W 
a ' 


(Z'p UOIQIaS B| ALOA) OT‘ = Q ¥B azuad Ee] aulWEXa 32 QI‘Q= D 32 S0'Q= DV & [eqos | adAy 
ap Anaiia,p xne} af 1af01}U09 Anod YAMA-H Ipoyjow e] ans aindde,s sajdyjnwi ssid ap INsouseIp 3.130N “SW 329 LIN ‘AS sosouseip 
$3] dade Sadyjnw ssid ap dNsouseIp a.1j0u JUeIEdWIOD (NEaINg sNsUdyD *S*f)) UORIAQey eB] ANS Salsgs SE 1nod saguUOp sap saskjeuy 


al}enpise. 9}ljeuUOSIes e| JNod anbiujyawesed UOU }$9} UL : UBIOH }@ Aou|QOW\ 


g neafqe | 


v8 


angdojereo ne X-100-Z1 oN ‘epeueD onbusnes 


“p’¢ UOTI_S 
RP] & WOOP jsa JI WWOD sa9UUOP Sa « IdIO[ONAI » 
yuaanad suaroneid say ‘Iaruuosies oid sroruaid np uoTda}9p 
BI Jonuaye jueAnod uonenualayip aed ‘onsouserp anou 
Jonbydde.p jueae sogsi[euUOsIessp soouUOp sop s0URpUd} 
R] JoUcIayIp op sjqeyeynos jso jl ‘someuUONRs Uo 
JUSWIDTTaNjIGeYy JUOS sonbruOUosS sanbiso,OUOIYD SoL9s so] 
dUIWIOZ ‘ayLIeUUONR}Is eT B.Nbsnf saaUsJOYJIp yUSTOS soguUOp 
so] onb jyuarginbor sonbynojduisse sjeynso1 soy ‘uUq 
‘saydnynur s}saq ap 
Sposa aOU op [eIONIO JUSUITS UN }sa Inb soNsouseIp SIOA 
-Ip sop onbyojduiAse souepuodapur.p sj3audoid ev] yueres 
inb 90 ‘syutofstp yuatos xneAou ap spoddns say anb sy0s ap 
g op ayewirxeul nojeA ve] orpuaid ap yuowonbneulsysAs suop 
-UPWIUIODAI snou ‘YJ daAe JUOWaTeINUaS onuTUNIp onbysne}s 
souessind vy anbsing ‘ayyeuUOsTes ek] op UONRSLID}OVIVO PB] Jed 
QdIp JUSWIIIVID so 1 yUOWOORIdWId,.| ap xIOYO I] “oJAND 
ud oOUI B sajduTs jo syINjUT yUOTeTQuIas Inb xnap Istoys 
JUSWIZTNs SUOAR Ud snoU jo UaTOneId un Jed saslNN ano 
Issne jualeLmMod npuedal osesn,p xneAou sone xnoiquiou 
op ‘UoUMUApIAW ‘juessind snjd juswsyeulsreul one yWeLmod 
IoMWEp 39 ‘AL 32 onbrenb xneAou say] anus s0udIayJIp 
ap nad anb suojejsuo9 ou snou ‘anbyeid ua nbionb ‘syey,nsor 
Sd] Ins soUapIOUI ouUN yUdWIOUTeYOO eB NeAOU Np xIOYyO IT 
‘((LOOZ) ETRY 1 STEAM “Aor qo “AoTpuly 
‘UOISY MOA ‘salolfnorred sargtuuostes saouanbey sap 
Ins sg{qld UONKsSI[eUUOSIeSap Op SONI Sop MOAddUOD mod 
g]9pour uN Ins sapuo} aydoidde oun Ins s}uUvINOD dYydIBYIOI 
ap xnevaey sop uojdussap sun mog ‘duas ke] MWOMODOVI 
ap JosesiaAua jretmod uo ‘uaIq no ‘ WWIYV-I[I-X suep 
sIoTUUOSTeS SAQII} Sa] JURasUeYD Ud ‘solejUdWIaTddns saguue 
Sop Ins jUessI] Ud ome as ynod nb 99) UoNesteuUOsIesap 
op SONY say Joyipour mod soaudoidde samsowi say arpuoid 
toaAnod op ule ‘saigruuosies saduenbey sayjonb ke Issne 
STRUT ‘dT[ONPISaI ayTTeuUOSTeS UN d}STX9 ]I,S JUaWaTNes uOU 
MOABs jNdA Inb IMasiyeuUOsIessp a] Inod jueyodum joadse 
un.p yde,S ][ “JorWUOSsIes JUSWOLIOdUIOD ne JUaNqUyUOS 
xneqsods soid sjanb Jowd0sIp op jouniod Jarwaid 9] 
[nas yo d moaypea oun }UMO} dU SONSOUseIP sap UNONe ‘gpl 
12 LIC ‘AS sonsouseip xne jUdWIaTqeIOAR} Zasse aredui0s 
9S opoyjoul aOU onb usIg “(O] B ¢ xneaTqQR}) a]UDDap Zasse 
}So douessind ve] 39 Wepuoye.s UO Janbne Inja0 3sa | adAq op 
SMO19,p sIquiou J] ‘sargTUUOsSTes satigs op adAq Joy UN INS 
"SO[QRSI[QN JUSWIOANLIOI JUBINOWOp s}eyNsa1 soy ‘saydninur 
soid ap uoNda}9p ap 3s9} a] nod (¢ nea]qe}) aT]Ie} eB] sup 
Ukd9 UILIDO UN dIB[RJY ‘p NevI]qQe} Ne sNUOUTEPp }S2 | SUTUIOD 
‘anbysyejs souessind ke] jUaWa[qeiIapisuod aloe ke 
dIQIULW op solgTUUOSTIes saoUsNbay buts sap sansst d smayeA 
$9] JouIquios Inod (] odAj op Maud ,p [eqo]s xne} 2] Jayo.QUOD 
mod suoneoydde soy omp-e-jsa,9) saydyjnur s}sa} say ims 
JINjLID}T] Ll OP Sexo snuuod s}e}[NseI sap suoAoy;dura snou 
‘sgjdnynu ssid ap uonsaj9p ap jso} op OLIeUZOS a] Mog 


€8 


‘Ol & [ xneajqri so] suep saquasoid asATeur p s}e}[NSeI Sa] 39 
suonenuns say red ganoid jso nagid OUWIUIOD JUSUISATIDATJO 
suUOT}OUCJ spoyoU ev] onb jIey a] ‘aNoeds 9] suep ojuE} 
-Iodul UOTNUTUIP/UONeUSWISNe UN Issne d]STX9 [I NO STRUT 
‘QALBOU 4S9 dIX9AUOD PT NO sed say JasIdap e spre sjuod 
ap onsouSeip oy “" *p ap ayeisua3 auuoy ey] ap o[noogp 
e199 £ sloyap ua snjd nad un suuasour es e sap sousnbey 
aun.p said sunueizopouied np suuaXdour oun.p uostered 
-uI00 aun puaidurod anbysyejs ap odAy 99 “uowoyTorduwy 
‘SuluuLpy-AoyNT, op saseyedsap sop anouay ey onb Ja} ‘onbidAy 
neXou UN .p apuodas daAuap ey] Jed sa1apuod surure1sopotisd 
np ouucsXdour oun.p jnojeo ne puodsal0d 9}1xaAUOod 
ap onsouseip a] ‘xneyoods soid sap onbysnejs uonsajap 
ap somneaou oyooidde oun yuoap sponse yuosoid oJ 


UOISN|IUDD) °S 


“SJCWLIOJUT Jd syURadeINODUS 
So WUOS sj}ey[NsoI say onb suosuad snou ‘alquiasud.] 
sueq ‘(snjd uou sed yuo} a] aU QZ NO /J) sold sap unone 
mod d mojea ap WuMO} ou AS ONsoUseIp a] onb sipury 
‘amnpoidai sed jusanod ou gy 39 LW $}S9} say onb [reap op 
Nnedalu UN PUNO; spoyjsut anou onb JausIynos op jyueyodut 
189 [I ‘OT‘Q= 0 Imod xneNoads soid sap uoNoa}ap ap 
ampeo0id ayou aonb smayjiow nod un sjeynsa1 sap yuouUOp 
AS 3° 8W ‘LW sonsouseip say onb ualg “HI 30 onbruenb 
xneAou soy Imod sarqe[quias zasse jUOS sjey[NSoI sa] 
‘(Sapueulay]e SoLI9s Sa] Inod jnes) 
QPOYIOW oHOU op sed JI suUepP | B IMOUaJUI oO e DOURpPUd} 
B sqoa}ap sod op UdXoU SIQUIOU 9] ‘sagsITeUUOSTeSap SadUUOp 
Sa] Inod SuoWIASIZAUT “‘UOSe} 390 op aysnfe ang sed ynod 
aU AS 389} JJ f sada}ap soid ap sag{a snjd uaAour aiquiou un 
JUSWIDTTaINyeU jINpold spotpous aou ‘OT‘O & SQ‘O ap assed 1 
[Inas a] onbsio7T °7‘¢ ap dpso,] op uUadour squIOU UN JUdANOS 
ympoid mb ‘Ag apomjoul ey op mnjao suep onb ampaooid 
anoUu op sed a SUep dAaI{2a snjd dnoonvaq jUSWATTaNIGQeY 
yS9_ Sayruiq Soules say mod sayoaj9p sord op uadow siquiou 
3] onb suOJON “s}UdIOYIp Zasse jUOs syI stored sieul ‘AS 3s} 
3] Jed sa499}9p xnao anb sautgur say yUOs sanbrput soid say ‘seo 
XNdIQUIOU op sue ‘oIpuaye A.S Hee} [I suUTWIOD ‘Q[‘Q= 0 
nesalu oJ Mod agaaja snjd souessind oun sare 69‘ 
® 99‘0 Op DUA Spo aou Nod (s}da1I09 s}e][NS9I ap ITRI0} 
uoiodoid ey] olp-e-}so,9) onbuidwa souessind ry] ‘sajruq 
SOLI9S So] INOg ‘J dA} ap sunaa,p onpusye uoodoid vy ap 
dOYOIMO} ek] Sup SLIdUIOD yUOS sed sad sTeUI ‘a}enbepeUT ang 
ynod UONSdLI0D kT NO (9}TX9AUOD ap sj}sa} say nod Q[‘Q= ” 
puenb) seo sanbjenb ajaaar ampaooid aHON ‘sa}tTuq seauuOp 
So] Suep sjUasoid jUOS siortUUOsTes soid sjanb sed yuonbrput.u 
S]I QuowmuepiAg nb usiq ‘sayqereduios sjey[nso1 sap yuouUOp 
SW 3° LW sonsouserp say ‘ sormuostes yenoods oid unone 
JUO,U Sogst]eUUOSTeSep Solis sop Uednyd e] onb jo sarstuuosIes 
JUOS So}MIq Solis sa] sayno} onb onbipur As 3jsa} a] 


600Z ulnf ‘a}anbua,p sanbiuyos | 


‘yUsUIJeNbape Sd93LUOS 919 UO SadsT[eUUOSTesap 
saguuop sa] Is Uoud e yuoWTeJA sed suoaes ou snou anbsind 
‘SOIZISSOIS SUOTVISPISUOD op Wes [] “xneNoads sod soy 
QUIUNT]S JUSWIDANIIIJO & UOTWWST[eUUOSIesap ev] onb yuesoddns 
ua ‘(Sougs op s[quiasua-snos onbeyd mod) onbuidwie 
[Te] ouN oUUOp « SaasI[eUUOSTesap sdguU0] »» 3}9}-U9,] SNOS 
SQUUO]OS Sp s[QUIOSUd,T ‘(PULIS }Sd JUSWID}D9LI0N sa}aI9p 
soid ap siquiou 9] onb amayjiow jueyne.p isa spoyjoul 
RI) SapousUI sop sylovoyjo.] op onburdwia smsoau oun 
amjI}suod Inb 39 ‘sa}99}9p 919 UO Inb sold ap uaXoul siquiou 
3] Issne suojuasoid snoN ‘onbudura souessind ve] op aials 
-Sois uogeuntxoidde aun suop jsa uotodoid ay99 3a ayITeU 
-UOSTRS BR] JUSUII}99I0 3}99}9p apoyjou onbeys onb sioj 
op uomodoid ve] ouUOp « s}991109 »» s}eI]NSdI ap [e}0} a] Ip 
jusWiaynNy ‘siotuuOsIes xneQoads soid sap ayuasaid ayja,nb 
}9 SIQUUOSILS JUSWIDATD9TJO 1S aLIas onbeyo aonb yuesoddns 
ua ‘(SoliIgs op sj[quiasua-snos onbeys mod) onburdwo 
sourssind vy] JUeUUOp SUID SIOpIsUOd a9 Jnad « sayeUTsLIO 
ssguuod » 939}-Ud,] e@ puodsauios mb sauuocjos ap aquias 
-Ud J “xne2[qr} So] suep sagjuasaid sed jUOs ou sa]]a,nb usIq 
Is ‘QQO‘O= J ddAe SOANROIFIUSIS JUDIE}. sSITTeA\-[eYsNIy 
ap onbysiyejs ev] op sMaeA So] SNOT ‘OT & S xNea]qQe} So] 
suep sojuasaid yuos anburduwia apnya 33399 op s}e}[NSaI SaT 

‘(amg du9s eT eB ,nb yuanbrydde_s au st,nb 42) 
exouu0d d moyeA ouNone 9}sTx9.U [I,nbstnd ‘aausIsuos 3sa 
IMIeA PR] BNas “gP 19 LW SONSOUseIP soy Mog °9}99}9p 919 
eu ord unone nb onbiput g epmua oun ‘¢ ‘p ‘e ‘ZT = Sf 
mod g/fu e yuepuodsauo0s f dsaae “aAnodep 39 
B JHeoiusis oid un sayjanbsay] mod sasstuuostes saousnbay 
sa] aJoumMus sN][I0 ap sanua onbeyo ‘OT e& ¢ xneo{qQr} 
Sa] nog (6661) Ae|pury 39 dnynog suep LOgp se Jo ‘agstyeU 
-UOsIesap jo 3}MIQ Saas sop (OE)YP aenoods uonewnss 
aun INS gpuoj 4Sd AS 3S} J] “dI]IQeuIs ap asaujodAy,] snos 
oy]euUOsIes ap souasaid ve] nod (7QQZ Ua Neamg snsuad 
SQ) SITeA-leysnry ap aysnqoi onbrysurered uou 3sq} 
J] SIOPIsuOS JUDUIATeBd SUOAR SNON “dTTe[TUIs UOTR}SIdI9}UI 
oun dade ‘algtuuOsIes ajuesoduios ev] suep suoTyeNjoNTJ sop 


ongojejeo ne X-[ 00-71 ,N ‘epeueo onbysneis 


J][!e} BP] sINsou gy onbysyeys ey] ‘ourgut ap 6 (g/6] ALO 
}9 URIYJOT) s[qQeIyWUSpI sed jso,U dLIas e] SURP d}ITeUUOSTES 
ey] onb juenbipur | ke sammauiadns smayea say ‘ayeursuo 
SUS LI SULP d|IGOUI d}I[eUUOSIeS PR] & JUDWIDATIPIOI 9]qQeIS 
auyeuuostes op ayQuenb ey amsowl / Jy ayyenb ey] ap ajomuo 
ap onbysnejs ey “uonesieuuostesasp ev] soide ja juRAe 
(AS) SI[ONsIA UOVOIJIUSIS sp ONSOUseIpP Np s}e}NsaI say anb 
Isure “QW 19 LJ Sonbysyeys say suojuasaid snou ‘anno uq 
“sotuuostes sold sap a3ejsidap 
ap s}so} soy Jonbydde.p jueae (seq snjd ywWoap ayjonsia 
UONVIJIUSIS sp dNsoUseIp 9s] Inod seo 9] 1sd vlad SUTWOD) 
QOULPUD} L] Op UONRNUDIOIJIP s[N9s oun nb anyoajja SUOAR U 
snou ‘sagsijeuuostesap onb Isule sajyniq saguuOp sa] mog 
‘Moyne Jord np saidne spuewlap ims snusa}qo a9 yUaAnad 
STO = Q mod sjeynso1 say ‘OI‘Q= Q mod sjeynsaI 
sa] onb 101 suojuasgid ou snou ‘UOISIOUDD ap ronos sed 
‘aonb ualq Is ‘sa[qe{quias s}e[NsoI sap yuastnpoid Q[‘Q = Q 
19 $70 = Q anb suojoN ‘({'p UONdaS eB] MOA) oId onbeyo 
® OLO= Q 2 S7O= Q NeIa Quad ap sjsaq soy mod [Inas 
J] NO 12 OTN 19 SOO= HY FB [egos J edd) Op Inaua,p xne} 
3] JURTONUOS Ud YAM A-H Wnpesooid ey onbijdde suoae sno 
‘HL neAou 9] onb sure onbryrenb neAou 3] 9dAe — VWWTUV 
-ZI-X OP [[X uoneoyioods ey] JuRsTNN usa — (sadsuUoJsuey 
-30]) SdosITeUUOSIeSap soguUOp soa] jd (SddWIOJsUeI}-30T) 
Soytq saguuOp saz Inod siatuuostes said ap s}saq say gyNoTed 
SUOAB SnoU ‘alias onbeyo nog ‘aLjsnpur,] ap jo adeuIOYO 
Np “MdLI9}X9 ddIOWIWIOD Np ‘sos ap SdJOWIWIOD np ‘[Iejap 
QP SdIOUIWIOD Np ‘UOTVOLIGR} eT ap simajoas say Ins yuRyOd 
19 AdOO.] op yueusaoid sanbruuryuq satias / | 39 sostesuey 
Saulas [|] ‘Ona ouOoZ ke] IMod saLias C] ‘sopueUraT[e soLi9s 
ZZ onb isute ‘[Iejap ap ddJOWIUIOD 9] INS SoUTROLISWIe SOLIgs 
OQ] 30 suoneyOdxa 39 suoleYOdUT say Ins SOUTeOLISUTe SOLI9S 
O] JuaUIaZO] a] INS SoUTeOLIDUIe SOLIDS Q] “UONeOLIgR] e| 
Ins soureououle Sauls ¢¢ jUdUUDIdUIOD SaLIas $99g “AdOO.1 
ap ‘sg Ja neaing snsuad “SQ Np yueUdAoId ¢g ‘sonbis 
-Q]OUOIYD Saas QE] SUTUIeXd JUOWOTeS9 SUOAR SNON 


seo ap sapnjyq 7p 


6660 8660 00°I 00°I 6660 8660 00°I 00°I 09€ 
00°I 6660 00°I 00'L 666.0 6660 00°I 00°! 887 
00°I 666.0 00°! 00°I 766'0 6860 766°0 6860 081 
00°I 666.0 00°T 6660 0560 7r6'0 7S6°0 €v6°0 byl 

8660 966°0 L660 L660 188°0 098°0 L680 LL8‘0 071 
HL anbyiengd HL anbyieng HL anbyiengd HL anbyiengd u 

(010 ‘sz‘0) (or ‘Or‘0) (s0‘0 ‘sz‘0) (so‘0 ‘Or‘0) 


sajdujnwi ssid anod yaAAAA-H nod asuessing 


([‘p U01Q99s BT] AOA) $389} XNIp sa] Anod sasi[ign 939 xnap sno} 

UO FLL 32 anbysenb xnedou saq ‘O10 = Q 12 STO= Q B 2anjeag aja & ajuad ev] ‘QI‘Q= 0 &I9S0'O= YM & J 2dA)j ap ANa119,p [eqols 
XN¥} 3{ JULI[OIJUOD Ua 9IIXaAU09 E| INod YAAA-H IPoyjIU | JURSTTIVN Us JUIWQUEI[NUIS S9IPN}9 9}9 }UO 9}IX2AUOD ap jd ajuad ap 
SdSOUsEIP sa] *saasT{N 939 }U0 SUDTHIdI4 OOO OL ‘1] ‘setdyfnu ssid ap ssouseIp aj anod aduessind ap uoHE[NUIs e] ap se}[Nsay 
p nesqeL 


a||enpises ey|jeuUOsies ej nod enbuyewesed UOU }s9a} UP : UBIOH }@ Aou| Qo" Z8 


andojeieo ne X-[00-ZI ,N ‘epeued anbnsneis 


L80°0 6710 ¢80°0 €£10 L80°0 Orl'o 
890°0 rau m) 690°0 9110 690°0 LITO 
LSs0'0 €60°0 650°0 L60°0 790'0 LOL‘ 
0s0°0 980°0 1S0'0 060'0 €S0°0 L80°0 
9600 9L0°0 ¢r0'0 0L0'0 Ly0'0 9L0°0 
HL anbyiengd HL anbyiengd HL anbyiengd 
(010 ‘sz‘0)=(9's) ~— (or0 010) = (0''S) oro=9 


s100 se0'0 610°0 Sv0'0 Z10°0 Zv0'0 09¢€ 
800°0 $Z0'0 800°0 970°0 600°0 1€0'0 887 
$00'0 L100 900°0 070'0 500°0 610°0 081 
£00°0 800°0 Z00°0 110°0 Z00°0 600°0 vr 
Z00'0 $00'0 Z00°0 900°0 Z00°0 900°0 0cI 

anbyiend HL aenbpiengd HL anbypiengd u 
(s‘0 ‘sz‘0) = (O'S) (so‘0 ‘010) = (0S) s00=9 


sajdyjnui said anod yAAA-H Anod oye L 


(Lp U01]9aS EB] AIOA) s}Sa} xnap saz Inod sastjyNn 939 xnap sno} UO FLL 39 anbyszenb xneAou sa “O‘O = Q 
39 SO = Q & aanjeag 939 & ajuad e] anb sipue} ‘QI‘Q= 0 & 39 SO‘O= DV & [eqoys | add} ap Ana119,p xNe} 9] JUL]OI}UOI Ua 9}IX9AU0D 
vy dnod YAMA-H 2poyjaw EL] JUKSIPIN Ua JUIIQUEJ[NUIS SJIPNj2 939 JUO 9}IX9AUOD ap 39 ajuad ap soNsOUseIp say ‘a4jnN0 UY “YWAMA-H 
apoyjau ET URSIN Ua O[‘Q= VD &I9 SH‘O= OD & 9[O4QUOD [eQol[s | adA} ap ANI119,p XN¥} I] IdAB JUAWIZId|YS JIPN}9 9}9 B 9}IX2AU0D 
ap 389} ay “saastyN 939 }U0 suOTHId21 900 OT ‘DY ‘sadyinwu ssid ap sysouseIp aj snod alle} eB] ap uUOHE[NUIS eT ap s}EI[NSIY 


OPSTTePI S29 % 06 
® amnatiadns souessind oun ‘pp = u aye) op suojMueyse 
sop mog ‘(soouuRQ{ jURIANOD saljansusur soguUOp 
ajdwioxa ied) uonesijeuuosiesap ey e apoooid uo] onbsi1o] 
anbneid ua sasAsasqo uo][HURYIa,p SoT]Ie} Sop Jueyuosoidal 
S92 QZ. = U Jeo ‘ayuepodun jusWoUIEnXe s}9LIdoI1d ouN,p 
138.8 I] ‘OT1 = 4% onb saynod Issne uoyNueyos.p saqiiey 
sop Inod surgul ‘au0us souessind oun yUaTIAaL (pf Neo]TqGQPR}) 
wOHRINUIS 399 ap sjeyNser sat (OT‘O “SZ7‘0) 1 (OT‘O 
‘Or‘0) “(S0‘0 *SZ'0) “(S0°0 O10) = (© “Q) Tones ke “(0 ‘Q) 
xnevsAlu ap somed anenb outurexd 39 9}1xdAU00-a}U0d 
yUIOfU0d 489} a] aonb aBesTAUD SUOAR,U SNOU ‘UOTR[NUIIS 33399 
Inod “uepuaday ‘ney snjd ajLID9p sadejo ston ua sinpgdoid 
QUIDUI B] SIOPISUOD SUOAR snou ‘aouRssiInd eI JoIpNys Nog 

"XNVUTWIOU xnedoAlu sap nod 
onbyanb arayyip ¢ neayqe} ne sdAsasqo onbudura ayj1e} ey onb 
JUOJ SUOSTLI S39 saynNoT, ‘sjuepuadaput yusuoNbyo}durAse nb 
yuos au soid buts say mod s}sa} say onb yso majovy ane un 
‘ IQUUOISUSUIP JUSUID}DALION yUSWIdANeUITXxOIdde sinpgdoid 
gun yuoutonbrun ynpoid YAM A-H epoujeul ey aonb suojonN 
‘uoT[WURYIE.p salle} soo mod ({ nesjqei) a0exd ay]Ie} 
oun apessod 9}1XdAU0d ap }s9} a] ‘anbrun ord un,p seo a] suep 
anb sipur} ‘ajjie} opuei3s doy ap sa ay1xaauoo ey ‘onbrrenb 
neAou 9] Mod ‘toy ‘EOE ‘B8Z= U D2AR O[‘Q= V Sed np 
USUWILX9 | SIJUOU 9] SUMUUIOD “(>D) JUDUTATNAS 9}IX9AUOD ap 3Sa} 
np uones[yN | sp seo oy suep snjd uou a}oexa JUsWIOUIEN Xa 
sed 4sa,U oT][1e} 99399 “(J adA} op INaLIa,p xne} a] siqtInbasap 
gjuod op jso} np uonestyyn,, onb sored) ayeurmiou mayen 
eR] op ayooidde -s aye} ey onb 90 e aipuaye snou sed suoanod 
au snou “(dD *S) seuTquios ainpgdold Re] ap sed a] SURp ‘IS 

“s19pour snyd suTUOD sI9pIsUuod 
BIOS CQ‘ & Mouadns Q NedAIU jo} ‘sjanprAIpuT sord ap 
aseq P| Ins a]UONd ap ssayjodAy,p js} onbeyo suonydajja snou 
anbsind ‘USUNMOPIAY “Weanosgp sa ord [nas un Is agyolai 


L8 


¢€ nesgqeLl 


ang ynod inb « soid ap aouasqe»,p oanbisse[d ssomjodAy | 
OP dIIJJIP IOl Sd9AIOSGO UOTENIIS RP] DUOG “sed SonNe.p suep 
aiAInsinod azosua suoAnod snou ‘sajafar1 yUOS S[ONpPIAIpUI 
ajuad ap s}so} suleyI99 Is ouIgUI ‘aonb 3sa UoNsonb ey ‘yuoprud 
a9 IO[NOA op sjqeuuOsiel s]QuIds |I ‘s[aNprATpul sord 
Sap nod 99yndexa 310s ampeoold e] ap ajuad ap 3s9} o1ed P| 
anb ualg SZ‘ 8 19 OI‘ & [esa sud 3sa (quad ap jsaq a] Inod 
NBIAIU 2) Q no ‘7 advj9,] 1Sd sIpucyoidde snjd uoneoydxa 
aun ajissaogu Inb ampgooid ay90 op yoodse up ‘¢ nealqe} 
ne soumMssI jUOS UONRINUNS 399 op sjey[NSoI say] 


‘quUepuodsa0D al9TUUOSTeS JoUNNbay ke] Inod saned 
-IJlusIs sasoise ajyuod oun,p spoAnodep x] e JUSWIOU 
-eynuis sed yynoqe,u Z adve}9,] onb 39 SAQeOIFIUBIS }Sa 
anoads a] suep 9999138 d}IXOAUOD eT JUOP sIOTUUOSTeS 
aouonbay oun IAnosep ap youuiad | odvy9,] Is JNO 
-lusis yuauonbysyes ord un aystxa [I.nb Jorejoeq ‘¢ 

JUBOYTUSIS 
yuowonbysije}s JuR}o sUTUIOD « oId » Janb spodut .u 
Joepoep op ue QH Jajofar sed ou suojreynos 
snou ‘tor.nb suojoN (SZ‘0 nO OI‘O & [eda Isa INb) Q 
neoalu ne ‘SH “[ONprArpul ayuod ap jso} a] Ia}nd9axe 
‘] odejo.J &@ WaAnoossp jyeoyusis sid yno} mog ‘7 
‘(COTO 
no ¢Q‘Q & [esa }so Inb) 1» nedarU Ne [eqo]s moaua.p 
xne} 3] Jajo.QuUOD mod YAMA-H epoyjeu ey] JuesIPHN 
ua ‘.\H ‘9\xeAu00 ap sajdnynu s}so} say Jaynoexy *[ 
: QYURAINS UOSe] RT IP S9}]Nd9xa 939 JUO (_D ‘S) d}TXaAUOD 
R] 39 oJUOd ke] IMs JUOWIDUEYNUIIS s9puo} s}so} Sa] ‘TeqoTs 
[ 9dA} op naue.p xnv} 9] JojonUOS mod YAMA-H epoyjeul 
eR] JURSTITNN US ‘O[‘O 19 SO‘O OP 0 xNeUTWOU UONPOTIIUSIS 
op sfiInos xne soyndaxd yUOs (Dd) JUOUWIATNAS 9d}TxXdAUOD 
PR] Ins JUeLIOd $}s9} Sa] “d}IXdAUOD BJ Jd aJUAd FB] IMs yUSUIDUE} 
-[nuIIs 9seq jso} uN.nb Isule yWoUaTNds d}IXdAUOD Fe] IMs 
QSEq JSO} UN SIOPISUOD SUOAR sNoU ‘dI][Ie} Be] JonyeAd Mog 


600z ulnf ‘ayanbuea,p sanbiuyoe, 


ondojejeo nv X-[00-ZI ,N ‘epeued onbysneis 


8r6'0 6580 LEs‘0 L€60 ZL8'0 109°0 09¢€ 
0560 90L'0 ZLE0 6r6'0 SSL‘0 Ler‘ 887 
106°0 L9S‘0 ZL7'0 £760 £r9'0 pS€E0 081 
66L'0 Ier'0 8070 9680 ZES‘0 L8Z'0 bbl 
0L9‘0 Seco Lrl‘o 8SL‘0 8er'0 L7Z‘0 al 
S60 =4 06.0 = 4 sso = 4d S60 =4d 060 = 4 sso = d u 
HL neson anbyienb nesoy 


(So‘0 ‘So‘0) = (2 ‘Q)-9t = g = 1 onbrun aid un snod asuessmg 


(['p UOIVdaS EB] AOA) $389} xnap sat inod ¢o‘g= 0D = Q jUESIWN Ua FT 39 anbyaenb xneAou saj anod ,uawgue}[nNus 
SIIPN}I 9}9 JUO 9}JIXIAUOD OP 329 JjUad ap sdsOUseIpP say] “({][) aed tuyap (Z)YP eepow aj aed sQuUOP js9 JANLUAI}[e asayjodAY, TJ 
*SaaSI[N yUOS suOIId91 900 O19 9/4 = g = UT OY ‘anbrun did un,p sysouserp aj nod aouessind ev] ap uOHE[NuIs EL ap sjeI[NSay 


¢ nes[qey 
0r0°0 1S6°0 900°0 1S0°0 776-0 960'0- 1€0'0 rS60 600°0- Z£0°0 796'0 €00°0 09€ 
€c£0'0 976°0 900°0- 1S0'0 1760 ZL0°0- $Z0'0 0560 700°0- L700 ¢S60 €00°0 887 
870'0 Z06°0 $00°0 €r0'0 768°0 1L0°0- L100 0760 Z00°0 7Z0'0 7r6'0 £00 0- O81 
$700 7680 900°0 Zr0'0 788°0 LLO‘0- $10°0 L760 110°0- r10'0 0760 v00°0- vrl 
810°0 888°0 $Z0'0 Z£0'0 7S8°0 $90'0- 800°0 €06'0 110°0- L00°0 £060 €00°0 all 
1D NBIAIN adAj-y189 4 auushoJ, «=—«-s«éNBAAIN ad4j-p189 4 auush0p, «= Q.:s«éNKAAIN ad4j-y1e9q auUucAOJ «=Q._s«NRVAIN] ad4j-y189q auusioy] u 
HL neAson anbpienb neAon HL nedon anbgienb neAon 
93IX9AU02) ayuad 


gm = g = anbmn oid un anod ae | 


HL ® enbyienb xneAou say anod juawiaiedas sgipnyg 939 U0 9}IX9AU0) ap 39 ajuad ap soysoUusEIp 
sa] ‘sagstqgn juos suonyesdss 900 01 39 9/2 = g = 1 By ‘anbrun did un.p dysouseIp 2 Anod aiie} el ap UOQE[NUIIS e| ap sjeI[NsIY 
] neaqey 


*suotlytj9d91 900 OT 22Ae NOC= 4 48a UoT[HUEYDI,p afte} BT “HL neAOu J] JULSI[IN Ud UaIssnes JUeIq 
yIniq dp a[nu asayjyodAy sun snos (q) (7) ah us ap 39 (e) (7) Ph up— dp UONNALIYSIP Vl ap sUWeAZO}SIY =| BANS 


(q) (e) 
HL-— 2}!x9Au09 ap onbyjsne}s HL — 2Jued ap onbrysyeis 
p Z 0 Gc t- p Z 0 c v- 
l i | ee ee ee ee ——__————————EESSSS-- (=) 
S = 
= 2 
s) s) 
a. a. 


aljenpises ey!jeuUOsies e| INod enbujawesed UOU sa} UF] : URJ|OH 38 AsO" 08 


anZojeie9 ne X-[00-ZI .N ‘epeued enbnsnes 


‘(satuag juaudojduy juasiny mod gqd) yuemod to; dus ,| 
Ins auas ey onb ({eoo] yenoads juoWoyIOduI09) s19TUUOSTeS 
anbrureuAp ouigul ve] opassod inb — D/P QUO 9] JUeSTTNN 
Ud JOURIQUIASIBIA Op UMUITxeUI Np spomjour ey Jed gysnle 
—(SZ)YV eyepour anou suouajqgo snou ‘ouoq ‘onbneid 
ud I9ANOy yeunod uo nb 99 ap juayooidde_s tnb no ‘saysiyead 
yuos nb (saneuuonrs) xnenoods sod sap ooae (€gQ6] 
uozieg Moa —onbiujsurered jenoods mayeumsea nb jue) us 
sa]qeisap sonbuosy} sajoudoid sas op 39 SuONe[NUITS soy SUBp 
UOHeSITWN UOS ap dyTIOK} Pe] op asned ve) (d) yp snssaooid 
un aInysuod op jiej}9 douessind op opme sanou op jnoof 
-qo,J ‘(sue 6[ & 9] ap ‘sadnoo0 sowmwioYy) JuemMod 10;dua | 
IMs oLI9s eB] B UWAYsnfe un Jed nuayqo (¢Z) YP eepow un.p 
}9 UaIssnes our]q IMIG ap snssadoi1d un,p med e suonnedar 
000 OI g[nums suoae snou ‘uTofuod jsa} aOU kB SadIDOSse 
souessind ey 30 alle} ey Joipmsa mod ‘uanbgsuoo seg 
‘uOnEINUIS eB] SUBP sLOdUIOD as saydnynur sjsa} ap syooidde 
anou jyUSWIOD OARS op jyURSSdID]UT yWOWOIOTPNIWIed 
QUOP }S2 |] ‘dOBdT]JO JSo UONRSITeUUOSTIeSap PI IS JOUTUIA}9p 
ap mA Ud aAIONpIss’ oyyeuUOsIes ke] Op uUONHeIO[dxd,] 
\so ayuepodum yuoure1A uoneoydde J ‘sayesapay sanbysye}s 
xne suoneordde sa] nod ajueyiodun a1sojopoyjaul oun 9yUas 
-oida1 amped 99 ‘SUUONUSU SUOAR,] SnOU sUTUIOD 4uTOfuOS 
}S9] Op sped aHOU op JUSWIApUOJ J] SMNSUOD sTONpIATpUT 
soid ap aouasaid ey op 3so} np oueusds aj aonb ualg 
‘UOTNTOSAI eT 1o}1en od saguuop op ayyuenb 
opueis snjd oun yresoyissooou mb Jenoods oid np souy snjd 
« UONVAIBSGO,.P SUOTIPUOD » sap eB puodsaui0s ynod snyjd g 
un ‘ uoT]HUeYSS ,.p a]]Ie} opuess snyd oun juaramnbar (sayuasaid 
uou syey[Nso1) J op safqrey snjd smoyea saz anb s[quias TI 
‘anno uq ‘(aijed dog jsa ayyie} es onb ye} ne aed ua onp jsa 
HL nedou np g]q1ey snd souessind ey anb suojou) su0} 3y)99 
ap samooeds sop inod jresorayoid 9] uo,nb uarq Is ‘samauedns 
souessind oun jo aie} oun mMoAe aquias onbryenb 
neAOU I JUSUIaTRIDUaH) ‘sdTqeuUsAUOd souRssInd ap ja are) 
ap sax3audoid sap xnap sno} Juapessod HT 19 anbiyzenb xneAou 
sg] ‘suUINSoI UY ‘ONSOUseIP Np UoNRsITeULOU ke] Op UOsTeI 
ud s[qvosI[sou \so oT[9,nb steur ‘, B ddXI} Jsd UONeAOUUT.p 
SOURLIVA BF] Op INgpeA eT ‘aInpgooid 390 suep ‘anb suoj}ON 
(7 neg]qe} J] OA) sduOUOId snyd yenoods oid un jueAe spuess 
snjd suoyjnueyos say suep (HI neAou) ¢6‘0 = ke Jenoads 
oid a[qiey un yuepassod suoyHueYse syed say suep (enbryrenb 
neXou) (770 op jueuea ‘pyonutod ke] ja uoT[HURYOE,| 
op IIe} ke] SOAR aJUDUISNe oNSOUseIP ayoU ap douessind ey] 
‘nAgid suWOD “xnendads soid sap yuaWIayoneAdyo 9] JUe}LAD 
Ud NO} BLWIXLUI aNQUI, Op MaBIE] e] WWMO} | Jeo ‘uONesTTeU 
-uosIessp SP SUONIPUOCD so] deAR sIqQueduUIOD }sod g ap 
x1oyo 99 ‘snd aq ‘Tenoads oid np suLoj ve] ap Jo UOTeSTTeOO] 
PR] Op 9pourwWod UOTesLoUTered oun suuOp |I.nb sored Istoyo 
9 © YP Fepoul ap “9/u = Y k ‘smMayney sayvs1ayIp 


62 


op ‘xnenoods soid sap aystxo [I Ip JUuoWeNNY ‘s[gpoul 
99 mod saleA xnap sayno} Jos (jH 3 (jH anb mMeYo 
1S9 [I £ S60 B S8‘Q ap JouRA suosIey snou onb ‘d op uaXowl ne 
sesLjourered jsa oid np ad10j PT “9 /U = g B 99XIJ OpuRq op 
mos e] 39 9/u = UN ev oid un doAkR ([][) Ud QUUOP (7) YY 
anbijsA9 sjapour a] saide.p ‘suojHUeYyss,p sol[le} SOSIOAIp 
mod ‘suonenuns OOOO] ep sieiynss1 soy suuOp Z neolqr} 
XJ ‘2 39 M ‘d ap xIOYO SIOAIp SaAe (| [) UOTAS UONR[NUNTIS oun 
® Juepsooid ua oid ap asesidap ap ampgooid oun ,p souessind 
eB] Jo1ojdxa suoanod snou ‘uanbgsuoo mg (dz/(,d 
+ [)@so9),_sod = Oy e gsiuitxeur ysa mb wl Pec 
+,,2@sood7—]|,9 = (y)f sed pguuop  3sa_ (TI) 
snssaooid ne o100sse anoods ay] ‘[u ‘o] > © s9UUOpP oxy 
souanbay oun B agioosse “0 UR] JING ap SOURLILA L] DOAR 


(11) ’3 = 'v(,7,d + gqosoodz — ]) 


Jed suuop (Z)YyP snssadoid ay anb sya} snssac01d 
So] yNPoUT sayeUIaye ssomjodAy,| ‘ord op sed aystx9,U ]I,.nb 
}S9 a}Isoduiod ayjnu ssayjodAy ayou onbsing ‘xneyseds sord 
Sd] JUSUII}JOIHON JoyNUep!l oUOP J9 ‘SQ‘Q= 0D = Q NedAlU 
ne ‘SH juawigueynunis yujafo1 ua yno} (jH Jeyelar sed ou 
suo}eYNos snou ‘yuoWasto9Id sn{ gq *d}1X9AU09 BI Op jo ajuad 
B] ap uIoOfuo0d jsa} UN IMs daseq douRssInd e] suONTeAd snou 
‘SUOTIIPUOD S90 sue ‘dId [nas UN,p doUASaId FB] Sp ISOUseIP 
anou op onbuidura souessind e] suosJapisuoo ‘ayinsuq 
“HL neAou 
a] nod anb onbrrenb neAou a] mod opus snyjd jsa ayy1e} 
PB] ‘aIX9AUO0D OP jsd} Np seo a] suep onb suojou ‘anno uq 
oP) “9 10 (7)"9 anua uONRaLION By e aNp ang a]quIEs nb 
UONNQLSIP ek] sp sLNOWIAse 9UN SUOAIASQO snoU ‘aT]Ie} ayEd 
ap suOT]HULYSE sa] suep ‘apna ajUdsoId eT SSNIII,] ouUTWIOD 
‘SoTqei[quias s}eI[NSaI sap guUOp yuo (Id1 sayuasoid uoU) 
g 19 11 op x1oyo saqne.p ‘anno uq “% ¢ ap jeuTuOU |INas 
ne juonbydde.s sanbiput 0 ‘Q xneadatu say ‘ suOTTWURYDE,p 
Soq]Ie} SasIaaIp nod ‘7 uonNdes ke] & soap xneAou xnap 
Sd] JURSITYN Ud S}e}[NSoI Sa] DUINSSI [| Neayqe} a7] ‘USUTWEp 
-uadaput soanyeaAd JUOS d}IXd9AUO0D Op 3a aUad ap sasay} 
-odky sx] SH 389 9uxeAuod eT Inod a[nu asgpodcy_| 
‘AUIQUI 9] ‘99IJI9A 489 ajuOd ey nod (SH anu ssetpodsy,| 
anb (uonRNUMIs 33399 ap sulosaq say mod) suosoddns ja 
NVIAIU NP dJOLS UONIUIJap ek] SUOAINS snou 4afeI ne mnoge 
sed ou suojeynos snou ‘ajuad ap asayjodAy,| ap seo a] suep 
‘anbyeid ua ‘Is sway “oIeIA yso_ (SH puenb (7 ‘{=/) (SH 
Jgjafe1 op aIqeqoid vy] JUL} SUIWIOD NRdAIU 9] SUOSSTUTJEpP 
snou ‘sed 90 suep ‘anb suoj}ON ‘WoWaATIedsal ‘9}TxdAUOd 
ap 30 ajUed ap s}sa} xne soId0sse XNRdAIU Sa 0 30 Q Jed 
suojussoidal ‘ueUajUTeW op Med Y “| oindIy e] e sqyUasaid 
juos saururerSoysty soy quop “(7)" A yo (7)! A ‘sastpeuLIOU 
soysouseip sop onbindwia uoynqiuysip oun juasmnpoid 


6002 uIn[ ‘ayanbua.p sanbiuyos | 


suonnedsr goood! ‘([9/% ‘o] ayearajut, ams aquao 
neAou un e puodsaui0s Inb 99) g/u = Gg 19 9/u =X 
d0AR HL neAou 9] JuesiNN ua ‘ogg = u ap UoT]HUeYOS,p 
g][!e1 (epueIs) sun Mog ‘oURTQ IMIG 3] Ja19pIsuOd ap Ns 
[I ‘sonbnoyduiAse suonesapisuod ap uosie1 ua 4uepuadao 
: SULIOJIUN JUSWIITedOT 1S9 sTeNIEdSs ayISUap eB] JUOP snssadoid 
ino} ‘sduiexe sed —juaulgueynuns sateiA yuos SH 39 
(JH slenbsa] mod snssacoid xnaiquiou ap aystxa [I WUaUD 
-JomMeN “sates os SH 39 (JH enb aos ap “g = (7)" A 
=(7)' * anb isure ‘] sura109y) np sasapoddy sey weysies 
mb uostlssnes oue]q mq uN suOjNUTIS snou ‘ouog ‘apme 
9199 mod 9]]Ie} ap sJomIqey uOoNTUYsp ek] OUOp suosodun 
snou }9 sajauuonnqiysip sajyaudoid say yuowonbuiduis 
JaIpryg SUOTNOA snou Tor ‘\H anu asetpodAy,] sdjofor 
sed ou suojyreynos snou ‘ajued ap osysouseIp a] suolapIsuod 
snou puenb ‘onbneid ua ‘Is sway ‘oursredas 9}1xdAU09 
op 39 aJUNd ap soNsouseIp So] SUOISpPIsUOD snoU ‘UOTe;NUITS 
31199 Mog ‘onbrun oid un,p aouasaid ve] ap oNsouseIp 9] 
Imod (nedatu) oe} e] JouTUIeXd,p yng Inod e suoNe[NuMs op 
g[quiasus Jomueid a] ‘suote[NUIs smoarsnid ayndax9 SUOAR 
snou ‘sonsouseIp sou op sdueuuoyied yy Jonjead.p UY 


uonenus ied apnjyyq so *p 


(9002) AOIAOP\ 3° URTOH ‘sueayq suep sojLIosp yUOS 
sonbuiduis sapme senne.q (7007 nNeamg snsuag ‘S'~) 
VINTEV-ZI-X oP 839 LI aiTenb e] ap a]9U0D ap sonsou 
-3eIp xne jd dJONsSIA UOHedIUSIS op ONsOUseIP np xno 
B S}ey[NSoI So] SUOTedUIOD SNOU 39 SdaST[PUUOSTeSaP SOLIIS Sd] 
onb Isute sayeulsLIO SaLi9s sa] suoIapisuod snou ‘ (qqoO.] 
ap Salas ¢g 39 NeaINg snsudd ‘S’'fQ Np saLigs ¢g) sanbis 
-Q[OUOIYD Saas OYE] Op ayins oun e xneNoads sonsouseip 
soq yuenbrjdde ua souessind e] 30 a]Ie} e] JUoWIonbutdus 
suorlpmys snou ‘sing ‘sIuIy suOT]WUeYoS soyT sUep 3S9} Op 
onbysyr}s e] ap souesstnd ap jo aye} ap sojauidoid sap sept 
oun jusuUop mb ‘uonefnuns red snuajgo s}ey[Nso1 suTey39 
suojuessid snou “IaousuTWI09 Imog ‘onbneid usa soueul 
-lojiod es SI9A JUeUaJUTeUI SUOUINO} snou snou ‘Tededs 
onsouseip np sonbuogy) sjyoodse saj addojoagp jueky 


sonbiuiduis sop” “p 


‘neAOU 2] SUEP gq UOTD91I09 
op mojovj a] sed suonbrydde.u snou stew ‘sogstjeuuostesop 
SOQUUOP So] SUOTUSIZIP snou ‘p uoNdes ve] op ssguUOp 
sop suoyeysniy! say 39 uoneNUNs ed sopms soy sued 
‘onejuawigiddns onbyeweweuw freaey un pombe: mb 39 
‘pUIULIAISP aN HOP GY op aSIsAUl JoLMO,] ap spuLojsueH 
Re] onb 4sa_ xud uos ‘uepuads>) ‘smoyoreyo suteys. 
B® ojuesinpes snjd aniered yeumod uorsus}xo oana9 

“QUIO }IOS Gp aonb UOTIpUOD e YUaUETeS9 seo 
90 B JUapUa}g,s aeUUONEIs sed a] od oxouUe,] & SagINOsIp 


ango[ejed ne X-[00-Z1 .N ‘epeueg anbysneis 


sanbnojduikse sajaudoid say ‘anno ua ‘ a]qeiSaul iso gp 
Is TUap UsIq yso InayeumMss,T +, |), 9 — I] = (Yq no 


(ne = (8)"69 


s1oye }so JURpUOdsaIOD M9}eUIT]Sa J 
‘uoNIpuod ay90 Jonbydde,p uaXour a] JOANON yWOUIaTIOPy 
suoanod snou ‘sarg1uuostes saouanbay sap mone squad jsa 
py onbneid ua.nbsmg -y neAou np yoddns np anjoxa sa 
O19Z sduonbey ev] onb Jomsse snou suOAaP snou usaUIaTIanN 
“Uasse ‘ a]qQeIs9UI UOHUOJ oun jos |, a-T|(Y)PV anb 
UOTIPUOd & IUIJEp Uarq ysa (3)"Q “SIOTY (961) PAOTION Op 
uoneIO[OSaI op ayooidde | ap y18e,s [I {yueAeIedne suTWI09 
X 9p swmureisopousd a yo 7 no ‘,, |), 9-1] (WI 
=(y¥)8 op uocfour ne owmsa ono weunod ayoads 
-opnasd ay “{’Vv} ap anoads aj sa f no ay a= OY 
=(Y)8 3sa {’x} snssao0id np ajenoods ayisuap-opnasd 
eT ‘oleuUOnRs JUsWIA}OLNs }so INnb ‘yY uUOTHUeYDI,] 
suous}qo snou ‘sagUalajjIp yuOS sadAtasqo saguUOp 
sa] onbsioqT (7 no [= p wWoulayjamiqey ouop) souepua} 
FT ep uonenuclgyip op apioy p mod “fo ‘Pix 
JULUDJUTLW JUOS S9AIBSqO SaguUOp seq aonb suosoddns 
‘apemoods 
gIsuap eR] op sajod say Mua}U0d sed ou ev UOdSe} ap dUIOQ js9 
neAou np yoddns 9 Is amey as ynad mb 99 ‘anoads-opnasd 
ne oysouseip a] Jonbiydde.p sjqeyeynos ano-jnad jiesas I 
‘quog (9/1 e aiatuUOsIes souUanbay arotuaid ey “Aorpnored 
ud) o19z souanbay e] ap soydoid sjanpise1 xnenoods soid 
Sa] Uonusye odiouud usa mb 4Yney-assed san] sop JUSWII] 
-Jatuassa juos sanbydde .(g — |) 19 g — | UoOneNUDIAIZIp 
ap smayeisdo sa] ‘IQ ‘sonsouseip say 1onbiydde.p jueae sioj 
xNop NO oUN SaasI[eUUOSIesap SadUUOP Sd] WeIINUIIZTJIp uo 
‘seo 90 SUR] SIO} XNAP NO SIOJ ouN saaIZ9}UI JUSWATTONjIQeY 
JUOS sogsTjeuUOSIesop saduUOp sa] ‘“UONesITeUUOSIesop 
B] ap a}xo]U09 a] Suep 4UepUsdad ‘aeUUONR}s snssad0ld 
un.p uoT]QUeYOS UN yUOS saaguUOp saz onb ssomoddy,] 
ms aindde_s 1asodxa,p suouda snou onb a1d0]Opoy}su eT 


SITIVUUOTZE}S UOU SIQUUOP Sep B UOTUIIXY p'¢ 


‘yney snid auWOEp YAMA-H eWnpeooid e] op 
QUaplAs UONeoOyIpour oun Jed 3U91qQO,$ }S9} 99 £ SargTUUOSTES 
ssouanbay salaruiaid ayenb say ims oxe yso inb « soid 
anenb e jsa}» un Jed gssaigjut snjd ang wexumod us1oneid 
a] Juanbgsuos Jeg ‘asouenbey ay00 e uONEgidio}UI oun 
Jonge, p aNoygip e] e otyed us uon e]9D “(9007 AeTPUT) 
9/US ap aietuUOsIes sousnbey swaInbuld ve] B SATVOIJIUSIS 
gjenoods asseul oun.p aseq ke] IMs sg}ofo1 UoWOIeI js9 
uonesijeuuostesap oun ‘onbyeid ua,nb suojou snou “uyuq 
aiAinsinod moanod mod seo unone suep 
yofal un & sUOISsNOge,U snou onb yneRJ [I No ‘argrUUOSTeS 
gouanbay enbeyo e ajued e] op syouljsip s}so} buts Issne 
suOjNdaXd sNOU ‘JUSWIaT[oIMeN “(exouuR,] e | SUWISIO9T} 
3] JINs nb UOTSsNosSIp P] JOA) 9}1X9AUOD ap 3S9} sp sonbyste}s 


9||ENpises ay\jeuUOsies ej JNod enbuyawesed UOU }S9} UP) : UJOH 3@ Aol Qo row] 


anojejye9 ne X-100-ZI oN ‘epeued onbysnes 


buto ap onbyojduiAse souepusdspur,| 3mua}qo mod gsijel 
-9u93 ang nod | sula10ay) 2] ‘sIo;y ‘ywIolstp yoddns 
un yuo inb—juowaAnsedsel g/ucg “ ‘g/u sargtuUOsTes 
saouanbay sa] Ins saqueo — Sp “* ‘'y xneAou burs suousid 
snou ‘uostel 31390 Inod ‘ja uoNeRIgpIsuod us sasud jsa} 9p 
sanbysne}s sop souepuadapul | yornbol spoysul 91199 ap 9} 
-IPI]eA BT ‘Sopoyjour sane xne JUsWeAneIeduI0S anbysye}s 
gouessind ey juawomejnorjoods arose oyjo,nb Iso (YAMA 
-H) 310qyd0H OP YAMA [egoys [ ody op MoLa.p xne} np 
ayooidde | ap adejueae,'T (a11xaAu09 ap asatpoddy,| ‘OH 
JOUTWIEXS B SUONIUN] SNOU snoU IS) 0 & o][ed9 UOITAUD SIOJ 
ap uonlodoid oun — sunone 9jsIxo ud_U [I,nb sroje sAnesou 
DUXAUOD oUN jURAR aIQTUUOSIVS dOUSNbey oUN sUTOUI ne_p 
gouasaid e] JOUTULIDJ9p sIp-e-}Sa,9—] odAy op smoLa sap 
AIMOUIWIOD SUOLIADP snou ‘ainpgsoid 399 jUeSITyN uq 
‘'y¥ S 1 mod 
7 soy sano, suojefar snou ‘sioje So(2 — g)1 s “Yd 
gjjonbe] mod apueisd snjd ey 72 op moyea eT ¥ OS ‘(GOO 
= 0 gduloxa ied) 0 neoatu ap ayyioads Jeqoys | odA} ap 
moaua,p xney un mog “77 sed sagusisap sayuepuodsa1i0s 
sg]nu sosapjoddy sot soar “ds Yds ds ds Vd 
UO]AS IOSSe[O Sa] B Jd SaIgTUUOSIes saousnbay burs sao op 
QUNdvYO & d}IX9AUOD ap }s9} a] nod d smogyeA soy JopNITeO 
B d}SISUOD (8861) BIOQUIOH 9p ampsooid xe] ‘suoNIpuoCs 
sou sueq ‘solgtuuostes saouonbay burs sap osunoryo 
mod (,)H sajjnu saseyjodcy sa] suoae snou ‘9}1XeAu0d ke] ap 
uonsanb e] B yuRUNT snou Ug “(S661 “p6z e8ed) S1aqyooH 
yo lueluag suep jap 10 (Qg6I) BIoquyooH sed ssodoid 
(ajp4 40142 asimAjiupf Mod YA AA) [eqoys | 9dAy op mous .p 
xne} 3] JajQnUOD op jURyoULINd spoyjsu oun suOJasI{NN 
snou 39 saydy[nu s}s3} ap SUOIPUOD sap SUBP SOUIWIOS snou 
‘yuanbgsuoo sed { (soid say sno} souTUNTS eB Issno1 sed e.U 
a]]2,nbsind) uonesijeuuostesap op ainpgsoid anou sajalo1 
SUOADP snou ‘syHeOIUsIS yUOs xneNHoods sod simoarsnyd 
no un Ig ‘(ayMuapl ane sed ynod su u souanbay ap oid 
31) anoeds np juoworda1 ap aip-e-}s9,9 ‘sei[e.p sawarqoid 
ap osned e xno ayua.p burs e JayIUN] snou suOAap snou 
SIRUL ‘JoIQ}UI UN yUa}UasoId sJoTUUOSIYS soId xIg ‘sogsITeU 
-uosIesop soouuop sap suep xneqoods soid op souasaid 
BI OP UOIDI}ap k] IP 9}X9}U0D 3] SUOIEpIsUOd ‘a]dulaxe Jeg 
‘soi dnnui $js9} op swisipesed un suep xney 
-oods sonsouseip op sfquiasud un JouTWexa.p jUeyoULIEd 
}so} uN suolgjoid snou ‘juanbgsuood Jeg ‘sMaT]Ie JIeopUsIS 
yeqoods xno19 un Jo[nuue jeLmod jlospua un e JNeoIUsIs 
jeyoods ord un nb yeas JUSTUAUOOUT,'T ‘sjanprArput [eyoeds 
oid op sonsouseip simaisnjd op oummos ey] jUoWadurts 
yIe1Is [I stew ‘ord onbeyo e un ‘spnaou simaisnyd yueyoduioo 
neAou un MOAIUOD suoLUMOd snou onb Med jsa [J 
“xneqoods soid simaisnid ap souasaid e] yUsUISUL]NUIIS 19}S9} 
SUO[NOA sNOoU NO UOTeNIS ek] JURUDUTeU SUOISpIsUOD 


uOTeSI[VUUOSIeSOp B] & UOT dde 
: said sanaisnyid ap aduas91d vy ap yulofuod ysay g"¢ 


red 


“SHXOAUOD ap 
}So} NP Nedalu 9] }sa9 0 NO ‘“(H) _@m > (7) up 
Is oid un aysixa jinb ampouoo 39 JH Joly ¢ 


nd-g 
(7) "wp queston 
Ud )yH eP IMoLIgyUT eIgepIUN 3sa} a] JoNDeXy “p 
‘yuasaid 4so,U 
oid unone.nb amyouos ‘uouts { a1AInsmod ‘(1]qQRI9 
-91d g0ueIg]0O} op [Ines anne un no) ¢Q‘Q0 <d IS ‘¢ 


“(IxI-) &@ 

= d ied sogyar d ya x sare “(7) Au up— 

=x js3} op onbnsnes ey op mopea osoruoid 
RB] OoAe agt00sse d maea ef iso d onb sosog ‘7 


Wy 
(pup 
juesyyN us yA ap JeIpeIIq 3sa} a] oney "| 
: QUBAINS USE} L] IP SIOTS 9INd9XI_S 19} ap aINpgdo1d ANON 
(‘uoTe[aI09 eT ap yUoWID}IOT[dx9 a}dwI09 sed suoud} ou snou 
}9 ‘SIOJ eT & UN ‘UdUEIedas $}S9} Sa] SUO}IeQ snoU “Io1}I[{duMIs 
Mog ‘asnjuod yustAsp d= simaea sap uoneoidiaqut] 
uepusdss ‘ayued op onsouseIp ne jUdUIO]]oUUONIpUOS 
QIXdAU0D Op dNSOUSeIP NP UO|NQSIP ev] JoOUTULIa}9p mod 
d9sI]NN 39 aUIT}SsO ang jnod UONRIPLIOD ay9<) ‘jUoUTOFedas 
SQINI9X9 JUSIE}D S[I.S DUMUOD dIXdAUOD kT Op jo aUOd 
P| aP S}S9} So] SUOIOIPPISUOD snou ‘(oxouUR | OP | JwugI09y 
=) 110A) Sd9]9109 jusuanbyoydurAse yustos ( Dd "hu up jo 
(7) Yau up— onb usig (py) ku Uf ap uskoul ne suO}s2} 
snou onb 90 ‘Iajofer eB suoyossyo snou anb ‘77 9sam} 
-odAy,] JoJapisuod suoAnod snou ‘sed 90 sueq ‘“a0ueTjUOS 
op gidep unone ooae ()H Jojofer suoumod ou snou 
‘jSso} op onbysyes ey Mod snuajgo jsd x aed yuoWUTESTyNsS 
IngyeA sun.nb ayno ua suosoddns snou Ig ‘oxouue | 
B sagynosIp sasoyjodAy say aonb sure eave snos ([ ‘O)N 
Io] op juawonbnojducse jso_ ( ‘ae Yhu uf<— onb suoAes 
SNOU ‘IQ “O19Z SP IANLOIJIUSIS IIIUeUI Op S1a]JIP apenoods 
dsusp eT op (ajuad) aigtmaid s9Auap ey onb amyouoo 
Inod ajuvoureAuoS yuoUTWesyNs sAnoid ap sed aystxo,u 
[I.nb jueyiusis sumwos 1aj91d1aqjut,s ynod ejag ‘a1Amsmod 
moanod imod agjafo1 ang sed yop ou nb (7 ayentur ay[nu 
asayjodAy oun suoae snou onb suojaddey ‘oid un,p souasgid 
RB] Op 3so} ne uoneorydde | jueUd}UTeU SUOIOpIsUOD SNON 


did [nas un, p aduaso1d Be] Ip }sa} ne suoneoddy 7°¢ 


‘uone[nuls red sopnys,p usAour ne [leap us snjd 
JUSUISLIOdWIOD 99 suOUTWIeXa snoU ‘p UOTIDaS BT YW “sonanb 
So] SURP So}IWIONXS xne Jnes ‘ayoold 4sa a]euLIOU 10] P| 
I9AR QOULPIONUOD FI ‘(][ sINBIZ) suonnedsl YOO OI 1° OE 
= U d9Av UdISsneS ou]G WNIq ap aT[nu asoyjodAy oun 
SNOS d9[NUIIS UONNQMsIP ev] op suIWIeIZO}sIY,| soide.q 
‘suoy[nueyss synod say suep 9}I]euLIOU-uOU op ds18ap 


6002 ulin! ‘ajenbua,p sanbiuyoa | 


UIv}LIDD UN 9sNed INayeI9UINU 3] DBA UONLI9LIO vs anb ja 
JUSWI9}US] ONb 9B19AUOD OU DOULA k] AP UONeSITeULIOU 
ve] onb juanbiput suoneynuis say ‘axouue,] e& sadinosip 
saseyjoddy sa] snos ja ‘uawaanoadser “SH 19 (SH 
snos (| ‘0)N 10] op juaswanbyoydurcse JUSUIOTeUISIVU 
xnap samo} quos (7) LAU up 32. ir Yu up— ee 
uq ‘OxoUUR,| B Sdd]NDSIp JUOS Ew nh op 30 Gah 

op sonbyojdurAse sajyotidoid saz ‘atidoidde jsa ]1 aurwi09 
[-UZ HOS u }OS JUOS J SIOLLWI Sap SUOISUdUIIP sa] NO 


‘ ng 
WO Cue) clin eae as Ol yh 
ng 1g 
aed), ell (7) “9 
jo 
‘ neg 
Z/uC 8h) zyP u WO er 6 shes 
ket Pai (Pain 
SACS A ow 4 (Dae 0 


Jed saguuop juos 
QIXAU0D BP] Op jd aJUAd eT ap sastjeWuOU onbysye}s sso 
oun no.q “Z/(,J)* @ eP eeLeo ouloRs ey Jed (7)*9 suo1 
-OsI[eULIOU snoU ‘OUD (6) JURSTTNN Ud JUOWITIOR] SOIJUOA 
ang med uonear one -7/(,1) 0 = 7/¥ (2). 
: QUBAINS O[NULIOJ Be] suBP | — UZ UOISUsUTIP op (,3)< 
HOS 39 UOT]MUeYDE,p saouURLIRAODOyNe sap [| — UZ UOIS 
-UsUIp ap ajooa uN {(T — 4)y ““(O)N “(4 - DY} 
= Y WO ‘ayUeAINS US] v] op [oJ0duI9} sUTeWOp 9s] sULp 
e[90 Jajuasgide1 suoanod snon ((8861) NIYD Hoa — £7 
op ajuepuodsaios a[eisaiul,| SIOA pud} _/ ap Q[eIdQ4UI | 
onbsind ‘stnbas ysa Z ap mmayoey AJ) °Z/(_J)* @ AP SOA P| 
Jed ayUasIOAUOD sIaTUeUT op seUTSa aya ynoed mb ‘([{xz ‘o] 
ms agkndde sa 3 Is) (_/) are 480 (7)°9 ap anbnoiduscse 
SOURUIPA PY “UONLUPA Md] JOWINSI,p oIesssd9U BIOS 
It ‘oatdoidde uonestjeuniou oun almmysuos mod “uepusdsd 
‘(exauUe | & Sa9gjndsIp) samejUsUg|ddns suoyIpuod souTeyII9 
snos sayeuLiou jusuIanbyojdurAse Issne steul ‘soyuasIOAUOS 
yusuanes uouU jyUuOS soanbysyejs suoNeUsa sad 
‘(g) suep (y)"A ap ayjaar 
aised ey aipusid suoanod snou ‘ajIxaau0d eB] ap 30 ajued 
RB] ap sanbysye}s samsoaul sa] Ja[noyeo nod uanbgsuoo seg 
‘VW X =X(8)Z,X anb aos op ‘y mayoaa yn} mod 
0=XN,X 00,.q ‘Zaydeol 19 onbinguiAsnue “o]]991 sa AV 
anb ja ‘zy1]da0] 19 onbunsUcs ‘oT[9e1 so PY NO NI + ZW 
= (8)qZ onb Moa op gpiovy so [I “(g) op queue ]994 
smo{no} elas y (3)%, XY ‘exatdwioo zde0], sp somjeu oun 
jso2(" y no 4 “dr, ano ined 3 no) (8)z Is awe 
quepuaday ‘sexajduios quos ("y)z 10 (" ty) anb [e1 
-9U93 Ud BIOI[NSaI [I YUSWIWIAPIA ‘7 39 | sajduroxa So] suep 
SO|NULIOJ So] IIOA a[duioxa Jed $ (g) suosijyn snou “(Q{) red 
SAQUUOP 9}IX2AU0D FP] ap 39 aJUad kv] ap soMsou! sap [a10du19} 
ouIvUIOp 3] SUep UONRUESsIdoI Re] Jo[No[eo sp uy 
‘sayuaiayjIp sonbisojouo1y9o 
Salas Sasnaiquiou ap e dajeda1 a1gtueWI ap soonbrydde sind 


ongojeye9 ne X-[00-Z1 oN ‘epeueD anbysneig 


‘gouRAR Pp soynpoid ang yoanoad ja (saytodxa sayduraxe 
sap nod ¢°7 uOMdaS PR] MOA) SIoJ a[nas suN.nb sasutTuLajap 
ang quoaiop au "%y yo "Sp ap sasioaut soos 
op sosuuojsuey =i anb as penonerpem SOUWIOJ 
sa] Janojeo sind “(" be Ace Si V)Z Zyyds0q. ap 
sooLyeul soy Jowwoy ‘(y) ,  (y) XA saouanbas 
sa] Inod ssuidoidde monsuo] sun Joutuajap juawnbrun 
SUOASp snou onb soled apouru0s jsa uoNRUaseidal snag 
‘(eonbyezpenb suo} oun snos s9udxa) jarodura} aurewop 
3] Suep UONRUAsgidal PR] SUOSITN snoU ‘UOTeUISA J JaTNoTed 
op UY ‘quod eR] op sIMsoWI eR] Op JNeSOU oUBIS 9] anbijdxa 
nb “(¢) op uoyesyyn. pusidwioo uonuyep anad 
Cc a = 


(OL) x ake x= = 


(Sacto 


r u 4 dy, 
a Gra = CARMI 


‘olnomred uq sande suep 7 red / yuowayduns juedse|duias 
ud dIIp-e-]s9,9 “‘UoNMNSGns op osyooidde oun juesiyyn 
ua { dIXdAU0d BJ ap 39 JUAd eB] ap Sansa sap sonbysye}s 
SUONeUISS Sap suOUd}qo snou 4uUenbgsuod Jeg ‘(exouUR Ud 
QNUOUI 4Sd ]I DWTS ‘3 UONOUOJ eT aUOD dyeDads UONes 
-9188 | red anjosai jso sWUUIeIZOpoued np s0ud319AU09-U0U 
ev] aonb suojou) seq snjd saginosip sayqrey SUONIPUOd sa] snos 
‘o<u puenb (/)*9<——())’6 onb anp-e-1sa,9 ‘saquad 
-I9AU0D JUSWIaTeIDNUNS JUOS dUTUTeISOpOLisd np sayeisg}uI,p 
aquas 90 ‘aqno uq (QO0Z BMEZTyeYy 30 TyONsIUe] MOA) 
sonbisojouolyo SaLias say Ins sIMjelayI] eB] SUepP OEP udIq 
yso nb Januonbay ourewop ne ja1oduray sutewiop np jassed 
op juesej9 usAoul uN dUUOp eIaD ‘UOTTHULYDE,p (senUa. 
uou) doURLIRACDOINS.p UOC} ek] & [esd (Y)Y doe 


LOSe (ae 


u—[=Yy 


66) (uu-]5 vay 
[-u 


Ie? 


Niles aX Xie 


aJUBAINS UOS} eT Op Peacoat ap onujuos 
agpueq sum sumo suosstuyap a snou “(| z/u] so T=S 
‘ujfuz) Jauno, ap saousnbay say sed tuyap JusWoTTomIqey 
JOS JomWep 90 aonb ualg ‘ouNUeIZOpouied og] sUSISsp J NO 


Uu 


= (VWI 


Dean ee u 
‘aa eign LP beets a EA = 

}So (3 a[qQeIB9}UI UONIUOF 93n0} nod) 
anbyeipenb suuoy ey aonb Jojou sed suosusumwi0s sno 


JJIXIAUOD ET OP }9 93U9d LY Op SANd}JVWIS|Y [°¢ 


‘QOULPUD} DAP SOITBUUON}S UOU SdgUUOP Sap & SUOISUd}x9 
sap suoynosip snou ‘p'¢ UoNOIS eI Y ‘UONest[euUOsIesap 
eR] suep ajyueyodun uoneordde oun oqtoRy mb ‘ayutofuos 
UOT}OI}Ep Ind] & UOIsUd}x9 oUN sUOUUOP snouU ‘¢°*¢ LOTS 
RB] eB Jo sold sap aT[anplAripul uoNosjep e] e uoNeordde | 
ap SUOINOSIP snou ‘7'¢ UONIeS B] Y ‘sanbyoydurcse saj3oudoid 


alJenpise. 9}!j;euUOSIes ej JNOd enbisjawWesJed UOU }S9} UN : UB|OH Je AoujQoW\ ow] 


angdojeyeo ne X-100-Z1 oN ‘epeued anbusneis 


sna] ap suONdsIpP jo sonbysyeys suoNeUMse So] SUOSSTUTJEpP 
snou ‘[°¢ uoKoes BT Y ‘asamjodAy,p s}sa} sa] suOIONjoaqJo 
snou puenb xnoSejueAr Blas Inb 90 ‘sayeuuou yuoWONby 
-o)duikse JUS sUOTRUIT]SA $90 ‘sayqrey SomeyUoUTddns sasoy} 
-odAy soureyiad snog ‘Jaoduia} ourewiop 2] sup Japnoyeo 
® sajdums jo sayuaBIOAUOd yUOS Inb 9}1X9AU0D FR] Op 39 aJUad 
R] op samsoul sop sonbysnejs suoneUss sop SUOALID9pP 
snou ‘uoydeas ajUasoid ve] YW ‘soguUOp so] soide_p suoneumsa 
sop oinpoid JuoWIayeI9UNS yey qi.nb uarq is f anoads 
9] sed suossteuuos au snou ‘apmyiqey.p “uoUTWApIAg 


anbysye}s s1s0[opoyjvzJ.“*¢ 


‘(saouanbas $99 ap a]]aa1 ated kB] JOIOpIsUod op WJNs ]I 
‘T'¢€ UONdas BT & Seq snjd suosodxa,| snouU sUTUOD ‘neaAnou 
op) oNsouserIp a] Inod saaudoidde z1dd0] ap saoujeur so] 


=0/(0) us 
op oe 


aIINsuod ap yUayouLIed snou sajnuLloj sag (| 
onb uonuaauod ey juesyyn ua) jewdoy,T 
eB] ep uoneoydde ap juanosep .f/ Uy = oe 

9 d/u = (0) TA onb OIG Gt ocho 


Ep eek Aa hs 
(uz — y)urs 


SUO}ON 


@z + yuis — me TES = (tn 
et ei Eu t iy, 

(uz — y)uIs 
UL Grth ¥ y JZ ? “a 
(uz + y)uIs fi ae {riyz} dxe —— = (y) Hh 
, u- ¥ u+ ¥ 9 7 as 
See {rlyz} dxo——- =" (7) ae 
G to y u-+ ¥ z ; ne 
crc eo) (righ doe = yy" 


sioye suuop (g) op uoteorfdde 7 


norG Th on ye wh aedrowcsdl ge.d\ etl (eae ey 
(7-y)uuis (7+y)uus yuuIsz aul 
; 7-Y T+Y y OT ¥ 
¥ SE FS uct | I Ci 


Lal 
(I + y)uuts 


Laat UP . 

¢ Ly seo net, EMSS ay 
Gi — y)uuls I ee 
‘ | ea aay eS, 

C — y)uuis (T+ aah att 

SUOUD}GO SNoU “(SaLIVd SINI] 39) 9ITX9AUOO PT 

9p jo aJU0d ve] op xneAou Sap dsIOAUI JOLINO, ap sdULIOJsue] 

eB] yueusid uq “YI neAou sed suolousisap ey] snou ‘asodxa,] 

ap ayns ey] sueq ([86] Aepseug MOA) senoods anguay 


GL 


aun suru09 suonbrydde | snou ‘tor anbionb ‘suruuepy-Aayn [, 
ap soseeoap sap anguay ey e onbyuapt jso UONOUO] aNa0 


UZ 
‘(ysoo—) ome (OV 


U 
yo ‘(yuls—) — = (Y)V 


UZ 
‘(ysoo + J) mes (WF 


: NVAOU UN INS S99dUOUD SUONIPUOD Sa] Sd}NO} WeJsyeS 
yins Inb xtoyo aT ‘snuIsod UOndUO}J 9uUN juURSITWN Us onbrenb 
neAou ne s]qQe[quias SULIOJ oUN IUd}qQo suOANOd snoN 


HI nvaon : 7 ajdwaxyq 


‘(saouonbes sao ap a]]9e1 otred 
RB] IAIOpIsuod op jyJNs II ‘["¢ UONdas ke] e ‘seq snjd asodxa js 
[I SWIWIOD) dNsOUseIp 3] nod ssgudoidde ZIdI0 |, ap ssomneur 
Sd] aINQSUOD op JUayOULIEd snou safNUIO} sag eudoH.T 
op s[99 eR] op uoneordde, FSpPhqaomios9p (u.d)/09€ 
=(0) 419 (.gL)/u0r7=(0) “A anb sun “7/Jy=y 10 


‘ of Oey cn ie 
YUISpS  YSO9pS YUISpT 
a ug 
eee oat (riys} dxa—=— = (y) 
Crist For thea 

YUISOST YSOOO$L YUISy7 

}dxa = sy: 
paca nee nls eas (4) 
¥ hf 


d = (y) "A 
)e yi} dxa— 7: (y) 


¢ ct zt + 
yuIsE — ¥SOOE ¥ UIs 


‘ “a4 et g Wd 
(sax yer yuls a rae i Nd 


suoud}qo 39 (g) suonbrjdde snou syanbxne 


uae! $iucaipacat turer -teaei file Tome 
YUUISpS YUSOoUPS YULUIS UpZ 
ANS 
idan GEARING ee = (y)*h 
YUsod ug yuuls,u } ¢7Z 
Cocercen nar 
YUUISY, YUSOOUNZT YUUIS -US/ 
al} 
seid ESM ri OMe Es (y) "A 
YUSOO UZ yUUIS uz | ZZ 
u 
( cf merlin imal oREETAGy os (y)vA 
yuulse yusooug yuuls,u | ¢C] 


( yf et! cf 24 ibe = = (y) ay 
yu uIs¢ yu sooug” yu uls zu 1Gh 


6002 uInf ‘ajeanbua.p sanbiuyoo, 


SUOUI}QO SNoU ‘(Sav SINI] 39) 9}X9AUOD FT ap ja ayuUad Fe] op 
xneAOU Sap ISIAUI JOUNO, op ssuUUOJsuey e] JURUAId Ug 


pus 

ICAL POR. re = (WP 
yu 

BON ey a ere = (WV 
Ug 

NCE O Se “— = (y)V 


: QULAINS SWLIOJ RP] QUUOPp (AT) & a SO]UTBQUOD Sa] $3}n0o} 
op uontsodun,J7 ‘onbnrenb oamp-e-jsa.9 ‘anenb ai3ap ap 
yetwoudjod neAou un Jarlgpisuos sed suodusumui0s snoN 


anbyaonb nodon : | ajdwaxy 


"SOSIDAUI 
IOLMO,] Sp SS9WOJsuUeY SIMI] ap 9}d1]dxa sULIOJ 9uN IUa}g0 
ynad uo,] onb 39 ‘apuoses ja aroTWaid sagauap sing] Jaynoyeo 
op glory yso [I.nb yso osejueAe Ma] “YI 19 onbyrenb 
xneAou soj ‘sojduraxa xnop suouTUIeXad snou ‘uUOT}d_S 
ayuasald ek] Y ‘SOSIZAUI JOWNOJ op soguUUIOjsuey soy JOU 
-Tunraygp Imod (g) JasITAN,p WyNs |! TexsUSs ug “apy sed 
4S9,u (At) onbstnd ‘saudoidde sed yuos ou yjarueq 39 yayeg 
op xnod onb sipue, YuouuatAu0d ([g6] Aapsoug suep 
sanosip) (HL) Sutuuey-Aodyny op jo Uozieg op xneAou Soy 
‘giduioxa Ieg ‘oysuap e] op onbujouresed uou uoneunjsa | 
Ins sINeIo}T] LT e JoyUTUdUIA say op JMS snou TI ‘ (AT) ke (1) 
SUOYIPUOD say JUOJsHeS Mb xneAou ap JNO} ouN 9}STX9 |] 


xnvAou op sajduraxq Ss ¢°Z 


‘(pH saneusaye | 
op Moar ud (jH Jojofa1 SUO[NOA snou 44so} 99 eB JOU 
-a]JouUoNIpuod ‘sind ‘77 Jajofe1 sed ou ploge.p suojNoa 
snou aonb jsa asamjodAy,p 3sa} 99 ap jamiqeyut joodse,T 


Rep OP ia Hoey SA = (f)"0: (SH 


0#()'0: gH 6 O= (/)'0: WH 


suoae snou ‘ord un mod asoyjodAy,.p 
4S9} 9] BWI20U00 Inb 90 uq ‘g < (/)’@ puenb odalBe- J 
xnaid uN suOssIUap snou ‘ajqereduios aroTUeWI aq ‘Z'¢ 
uoldas eB] @ Sanbysye}s s}so} sa] SuOIOUTUIexd snou puenb 
yUapIAs snjd eIpUdtAap P1990 { 9}1X9AU09 PT SUOCUTULIA}9p snou 
onb sind ‘9 = (/)"69 Is Joutuuajep red suosusumu0s snou 
Je1gues ua ‘onb 29Pl.] ap anosap jayuanbas jsodse J 


0 > Y)"o "39 = (/)"0 


Is anoods np (p e yoddei red) adoise- J 
oid un ysa nf anb suosip snou ‘sing ‘[ed0] Wos aydeds 
np ana ap jutod ayou aonb suoyieynos snou oyjenbe] suep 
omMsoul e] ap puedeap mb uoIsidep ‘oMjoxs suOIISsp snou 


angojeied ne X-[00-ZI ,.N ‘epeued onbysneis 


anb saouenbey sop uolouoj ua IsIoyo jsa_g { (xno uN no) 
oid un 19]99}9p suoyeYNos snou no aydeds 9] sup }oIpUd,| 
ap uonouoy us ‘souRAR.p 1 op IMa[eA ke] sUOSsISIOYS 
snou eeira SPOUOUP $99 JURINULIO} Ud, nb suousijnog 
A ii Oudeu( f v Yo ap jofns ne sjanuaigyut sgou0us sep 
JO|[NWUOFJ & juejeambe SUIUIOD S91NPISUOD OQ Jnod aT[aKUENb 
-9s sysoidde | ‘ord un,p souasgid e] JolueA op uyy 
‘soouonbay op opueg sul9u 
RB] IMS SOIUTJSp d}IX9AUOD Op dINsoUI 9UN ja ajUad ap aims 
-oUI oun ‘JUsWoT[aNUONbes “JouTWIeXS JUSWA}saytueUI OUOp 
NJ [[ “OANLSOU JUSWD}OLYs ‘Opuodas S9ALAP BS 3d JT]NU 4sSa 
sigtwisaid s9Auap es onb ajueuTUUaj}9p d}audold ev] apassod 
yeoo] Wnwixeul un ‘oyeusoyujur osAyeue., op apo 
SUISIXNSp op sUjOWIOES ev] sue “(QUOJOUOW }sa NO) xnaID 
un e no ord un e puodsauios aydeds np suuop ayjeAraqut un 
IS JOUIWLO}9p op jSo eI9UNB JNDAGo OHON ‘a]TeI0] aTeNOeds 
SUjOWIOSS RI OP S}UCUTUUIAJap sop soaleUOWIS]9 sajueS 
-odul0d say] juajuassidar JUusWIoIMLIDJUR sayLIDap sayeyoods 
QIXSAUOD BI Op jo quad e] ap SagBaIse soInsoul say] 


ssid ja xnoigQ 77 


‘axa{duro0s sa aed es onb j1ej as [I,s (y)"A ap aj[aer ated 
RB] onb Ialapisuod SUOASp dU snou ‘onbiysuIAs souUaNbes oun 
iso (y)“A onbsing ‘soudoidde xneXou sap sasiaaut Jono J 
op sasouuojsuey sap souanbes ey] yueyoes “(Z) JUeSTTyN us 
QIX9AUOD PT Op 30 aJUAd eT Op SamMsoUW Sa] Jaynoyeo sucANod 
snou ‘{’v} snssaooid aj mod (y)/A yorodwa, surewop 
ne dAje[aI uoteULIOjUI., suopessod snou Is ‘ou0q 


V J Sy 
*k {riyn} d ZOD UT 
(8) OD UA ERS el a 


Ss 


(uz / gy) rig dx Gp M iat 


£ 


(uz / dy)" 4 krlys} dxo at 


(4 


lI 
— 
a 
—_ 

x 


(12/4) Arg dxo = Gian d 


syns fr ounos ("Sy)z 19 (" Sy) z (" y)z (" Y)Z 
aimysuod suoAnod snou ‘9}IXd9AU0d RI op jo ajUod eI ap 
sguUOQUSUISNs xneAOU Sop dSISAUI JOLINOT sp s9uULOJsuey 
RB] jueuaid uq “yp neAou np mued e (y)’A souanbgs 
B] JOMUIDNJ9p op xnaseyueAe ang WeuNod [I ‘ouog 


o-=Y 
(1) CY Men Re =e CEO 


suo} ve] snos (f)"9 
9110991 suoAnod snou onb (7) soide.p suojou snou ‘uyUq 


eres Ra) 


a|jenpises ey\jeuUOsies ej nod enbuyawWeJed UOU }sa} UF] : UB|OH 3@ AOjQoyy pl 


ango[eywo ne X-100-Z1 oN “epeueD onbusneis 


J 
G mat) =). u9T Ou: 


Jed sQuuop JUOS Sd.ed Sd] 
‘{u ‘9] > [z/d + 1 ‘z/g — 1] onb juesoddns us ‘ouog 
‘sojuepuodsali0d sasIdAUl JOUNO, op soguLojsuey smo] 
19 ‘xneAOU $90 ap SaLIvd Sd] SUOJOIOPISUOD snoU “pIe} sn{d 


Jed souuop juos 
mb ‘" 9p 49 "Sy ge suossozgquI snou snou ‘ascTeue o19TWOp 
ud uoumuepiag ‘sonbnsneys sIMo}eUIsS sOU sURp eIOQUS 
g]]901 ated es g]nas ‘ourfegy ud stew ‘saxajduiod sinoyea 
B® UOTOUOJ ouUN 4s "Y anb sJayejsuoo ap yueyqnon ane 
jnad [J] “siatyUa UOU sJUsUNSIe sap e (¢) JURpUd}9 Ud sonud}qo 
JUOS sSUOT}eIa1 sao aonb dyI0s ap ‘salgMUa UOU sINo[eA sop 
e “A yuowlayjamiqey suonyeaAs snou anb suojon ‘sajqeiedas 
‘suas UIe}I99 UN Ud YUOS 11 9p ja g ap sjayJo Say onb oplos op 


(9) “(uz / dy)’ A{rlyz}dxa = (y)” “VA 


anb suoA0a snou ‘sayqeuea 
So] JURTIJIPOU UY “U NO CQ ke [e389 4so 11 IS samsour ap sadAy 
S90 drImysuod sed suoAnod ou snou aonb suojON (AT) ® (1) 


suompuos say yeysnes "%p neXou 21 10 { [uz ‘o] > [zd 
+1yc/g=n onb; inod: gi 12)¢ s gs 30). IZas 
aonb Jasodwi suoAsp snou uowiaysayuepy ‘[7/q + 1 


‘7/G — 11] aouanboy ap opueq ev] ap sioyap ua [NU jsa }2 


{cr - ¥) ars - (Y) Sey 


red 1uygp sa" *p ‘sioty “0 = (¥F)¥ 10 

(1) wWeysnes nb [u ‘u—] spueq ej Ins uyop p sed neAou un 

Jastpqn Jed JadusuTUI09 suoAnod snou ‘a]e.yoods 9}ISuap ke] ap 

uOneUITsS | 39 onbuysueIed UOU UOISsaIs9I Be] Suep xneAoU 
a AG gjuepuoge oimyesony e] ap yuendsu snou uq ("4p 

8p yg|Ng]e9 suorssind snou onb yuesoddns ua) sagauap 

SdS 9p a][99 uOU jo ‘f op doURSsIeUUOD ev] anb jyUareInbal 


su sayja.nb sored ‘sapowmuod yuos saynuLIo} sad 
} 4 dp, 4 dy, 
(6) (/) “e=(Y) "e 
neg n° 
Y). Heh Hoo 
€z 


suoua}qo snou ‘(p) suep orped red uoneis9}uI oun .p 
12 (AT) op uskow ne JueUQUTeYy ‘neAOU Np 92[e}0} oSseUI 
BR] ap juowonbyeuloyne ajdui0d eipuay inb uoneseuuoU 
aun suosatojdura snou ‘prey snjd onb sored ‘oyun ] 
e ge8a ylos “py op aye}0} aye13a}uI,] onb sed suosoduit,u 
snou anb sauStnos ap yustauoo qT “87 neXou np 1ojdu,| 
zed ayipow sed eias ou f suep oid np juowsoedus,| 
onb ynuerd (1) uompuos ey 9 = (7/g Fu)" 8p (al 
pa oya0 ms rf ap mone onbinouiks yso "Sp (im 
‘(7g +1 ‘z/g—n] opueq ef op sioysp us my so "8p (n 
:[u ‘u—] ms _} suep uoNdUOJ oun jso "dy (1: soyueaIns 
soaudoid soj yuede xneXou say suoigpisuod sno °" 8p 
Jed sgusisap yuonbgsuoo sed yuoIss syuepuodsaui0s xneAou 
sy 9 ‘[u ‘o] >[z7/d + ‘7/d-u] opueq e] ms f op 
DIX9AUOD LJ Op 39 aJUAd ke] Op oIeUIWIOS oMsoU uN yUOUUOP 
Sd][PUUONDUO} Sdd “SdALIOP 9[NVs ouN sUSISOp yuIod onbeys 
no “(/)’9@ 10 (/)"9 sed sarmyap ouop juos 9}TxdAU0 
e] op Jo ajuod ey ap saa8arSe samsoul soy ‘ayjauuonouoy 
ane0 op «neAou» a ogfodde vias Pp uoNoUoy eT 


Hats 
(p) AGANG REL Ardlpre saan a 


no ‘9 a]JauUOTNOUOCJ 
oun.p [eisues juisodsip np udsour ne geIsq}UI 3190 
SUOUSISS9p SNON ‘spueq ke] IMs uoNneIs9}uI,| mod sgudoidde 
uoneunxoidde aun asstumoy Pf onb uonIpuos e ‘apueg 33399 
Ins yoedusos yoddns un apessod nb Pp uoTdUOJ oun BUD 
Jo139}UI jnod UO USWOANRUIDI[Y ‘Ssousnbey op opueq oun 
Ins uoneisajut red JUdqQO,S SSALI9P S20 op sasaIse sINSoUI 
aun { aspuodes }9 sIgTWIOId Sa9ALI9p Sas op UdAOU Ne 3}L09p 
ang nod _O suep UOTOUOJ oUN.pP eI] SLNQUIODS eT 


J}JIXIAUOD EY BP Ja ajUad ET Ap Sainsayy LZ 


‘(,O sed suonouoy 
ap soedsa 390 a8oiqe ud suoIajUasgidaI snou) 2,qeAuop 
yUDUINUTUOS SIO} xnap jsa_f aonb suosoddns snou ‘spore, ] 
ap ouns ey sueq “o> |(y)/A| Jy] *-£< Is e[qeaup 
JUSUMUTJUOS SIO} P UOTOUOJ sun yso f “(Z) salde.p ID 


aera 
Wry nP C08 | = = = Nik = B)"Z 

onb jmsua_,s 
Il (8) z sed 7A ev ag1d0sse z[doo], op somneur ey suoUSIsap 
snou Is ‘ayno uq ‘sendeds ayIsuap oun mod juswonbrun 
sed uou }9 38 9] qQeIZQ\UI UOTOUOJ 93n0} mod a<IfLIDA 
so 3 yo 7A anuo uonear one uoulayJomeN ‘oyns 
eB] Suep sosudar smaisnid e suosast]yn snou onb uoneyer oun 


(2) WP wrCOF "f= = Wis 


600Z unl ‘ajyeanbue.p senbiuyoe, 


QUUOP SIOAUI JOLINO op souLIOJsuey 


e] onb ymsuas qT ‘[u ‘u-] 3S Y 10 I-f = 17 deae 


(2) mn 2k DZ = COS 


Jed aguuop jS9 ‘a[qeuIWOs yUSWINJOSQe j10S 
(y)/A4 soueLeAocco;ne,p uOTOUO] e] anb uOMIpuOD ek sTUTTOp 
ua1q ysa nb “(y)f afenoads gysuep ey (“x “x 
‘y) = ¥Y wuonejou ke suosyN snoN ‘ay[nu suUaXouT 
op areuuoNnes snbyseyso}s snssoooid un,p nsst uo[WUeYdE 
un yso “yo “y “y “oseq ne uonenualsyip oun 
yo sogudoidde suoneunojsuey sop soide.nb suosoddng 
“so,duuts 
xneAou xnop ‘sayodxa sajduioxe.p ann ke ‘suojuasaid 
snou ‘¢°7 uoKdes e] e.nb sipur} ‘said sap uoNda}9p ey mod 
SOINSIUI $90 dp VdUSUTAd ke] Bp SUOJNOSIP snou ‘7°7 UOT}DaS 
RB] V ‘SuX9AU0D PI Op 19 aJUAd Fk] Op Samsau sap s}daqu0d9 sz] 
suoyuasaid snou ‘[°Z uo1jdes BP] Y ‘sonbysye}s soinsoul sap 
SUOIOZESIAUD SNOU “¢ UOT]DES P| e ‘prey snjd ‘ (¢g6] oueseg 
Jo UOIMON Op syooldde,.] MOA) sajstuUUa}9p JUOWIOT}US 
yuos nb anoods np 9}1xaAuod ev] ap 39 ajUad e] op sainsoul 
Sop suoIspisuod snou ‘uedep ap jyulod sumw0d ‘ssIpmes 
onbisojouolys sues ev] ap (anseds no) sjenseds 9syIsuap 
R] dp suj9UIONS ke] BP sUONdSIP snou ‘nar] Jomusid u_ 


3.1399ds Np 3]¥IO] 91.199UI09S BI ap aANSIJA, °Z 


"SaAnaid ja SdUI9IO9} Sa] Sno} ‘axouuR | B 39 ‘sUOTSN[DUOD 
sou suojueseid snou ‘¢ uoKdes ke] Y ‘Say[anplArpul saouenb 
-91] Sd] MS saseq s|sa} sap oNsoUSeIP [nes uN Ud JOUIquUIOS 
Inod (gg6] B1aqyooH) satdyinur s}so} saj Ins sINjeIDVI] PY] 
ap sei) s}daou0s suteziao suonbrydde snon ‘(qqoo) senbru 
-0u099 JyUaWIaddojaAap ap Ja UONeIEd009 ap UOTesIUeZICO,| 
op 39 neamg snsudd ‘S'f) Np jeUsAold sanbisoj;ouoy9 
SUIS YE] IMs yueyIOd aTJoyO9 opueId e sed op opme suN.p 
UsdXhOU Ne Sapoy}oU sou op d}I]N.] anno usa suoyUOWISp 
SNON ‘}Sso} ayou op souessind ev] ap 30 alle) e] op ‘Tuy 
uo]]WUeYSS US ‘UONdLIDsap oUN yUaSsTUINO} SUOTe[NUIS sap 
£ TeSSd.] B IISO[OpoyJoUW ve] SUOYoUT SnoU ‘p UOTDeS eB] Y ‘sold 
So] JUeWIZ9U0D sonbnsnes sosetpodAy so] J01S9) USWTUIOS 
jy sonbysnejs smMayeulnsa so] Jo|NUOJ JUSUTUIOD suONnUOUI 
snou ‘¢ UONOIS ek] Y ‘sISIOyS yUoWesnousIos xneAoU 
xnop op uaXoul ne suoysnyt say 39 spomeu 990 op sanby 
-eWloyyeu suoOU sa] suoddojaAgp snou ‘Z uOTdaS BF] VY 

“uoTTHURYOY, | ap eTTTe} 
®] op spuodep apueq op mosey ej onb sues siew ‘neAou 
np spoyjouwl e] sed gsstj owuesopousd un.p suoneunso.p 
uO} ke] JWouUId soja UoWONbHeWoWeEY] ‘JUSTAUOD 
JI auTUI0D sagslTeuuOU ‘ayenodods oIsUSp ke] OP d}TXOAUOD 
PR] op Jo ajuad ev] ap saadaI13e SaMSoUI Sap IP-k-}S9,9 ‘apo 
juasaid a] suep asodoid oysouselp ne yWNoge UOTXaTJeI 990, 

‘Ov ap pueis JUSUIa|qQeUUOSILI BBeUISIOA UN sUep sATedOU 


angojeyed ne X-100-Z1 oN ‘epeued onbnsneig 


OS 3}IX9AUOD ke] onb ynej I uanbgsuos Ieg ‘ord un amNsuod 
Inb 39 ap SoAIMjUT snjd suonouU sap s100sSsIpP juRI9 Ud 3NO} 
‘({) alejsnes jnod su0j}OUOU UOT}OUO} ouN suEp ord yad janb 
syoduit.u onbsind “°y anbrun jutod ne f/ ap ayeunsayuyut 
SLNSWIOSS LI IOUTWUeXS,P JleU d[QUIVS |! ‘IYOgTJeI UdIq NO] 
‘suas un Ie ydadu09 9] onb mod uoyvoriusis auinjiao aun 
JAAD ZANLSIU SQ WOP spuodes ddAUap eI USWa}sayTUePY 
“gpuodses jo sJoTWIaId SagAUEp so] jyuousIsa=p f 30 f no 


(1) OSs 29 = CNS 


anb aja} °y aousnbay 
oun jso oid un “(s0ueLIeAODOjNe.p UONoUOJ ke] ap apider 
yUSWIWeSTNs sduURSSIOID9p ouN,p udXsow ne dse1jWUeNb 
ag ynod mb) asst] UoNSUOJ oun oUTWIOD =f aTeNeds a}IsUap 
SIVIA B] SUOIPISUOD snoU Ig ‘JHeOyIUsIs JUSWIONbYsHe}s 
aid un.p souasgid ey] Ja}so} e@ suUOYdINYD snou enb siew 
‘(sorgtuuosies soouanbsy xIs saz) 1919}UI,p seouanbay sey 
elap suossieuuo0d snou onb suds 39 Ud ‘aJUdIO]JIp JUSUIIII39] 
yso_ oyooidde anon ‘anoeds np xneoo] simostutxew 
S93] INod s}USSIDAUOD SINaJeUIT}Sa sap yuoIOgeIa Inb “(¢g6T) 
oursed 19 UOJMON Op INjao 4so gyUaIedde ajo ane Uy 
‘oid un p souasaid e] op samejuswe|ddns 
sonbysyejs soanaid sap ILyjo mod gsijyn ayg ouOp ynod 
nb soid sap uoydajap ke] Inod onbysyejs uoNeoyUsIs op s3} 
un suosodoid snou ‘ajore Juasoid a] sueq ‘20Y po aistueW 
op sijqeig jUOS « dTJONSIA UOHeOYIUsIS >» el IOUTWID}Op 
Inod sjinas saj 32 — asoyjodAy.p 3so} un ke agIoosse sed 
yS,U a]]o,nb aip-e-jsa,9 — onbysyeys uou jsd uOTedIJIUBIS 
Re] :onbysynejs ajuesoduiod ounone juawieiA apodwos 
ou aT]a,nb yey ne jus soyUMN] sas op oun,T (7007) Neamg 
snsua9 “SQ NP VWWIV-ZI-X ewuuesgoid 3] suep ayjamoe 
amoy,] e@ sonbijdde sa spoyjow aad ‘pues JUOUTWIeSTyNs 
so peog,] Is o1d uN dUIWIOD dDIapIsUOD jsa ja ‘saydold 
snjd so] SouIsIOA sImajeA xne dsgreduiod jso (a[qeiAno 
Iof ap souenbsy no) asigtuuosies aouenbay onbeyo 
eB anoods np moea ke] YUoUIOT[aAIUSSsO — « SyROIJIUBIS 
JUIWII]JONSIA » soid sap J9ANON Inod jissoisa1oyne aysods un 
Jasyyn,p yuasodoid inb ‘(6661) Aeypuly 30 dnynosg ap aT]90 
yso yofns 90 eB ayueyOdum snjd e] omauajue UOHeoT|Gnd eT 
(P86 1 JOWTTH 39 [19€) e]gpour 
UN IMS S9puo} UOHeRSITeUUOSIeSp op Sani op yuowoddojoasp 
op xneaey Sop uoIsojdxd,| NO,p — auipuuoYHIsS yyeUUOSTES 
RB] op syoodse suteyisd JOUTUIIIS.p s[qQeyeyNOs jUOWOTeS9 
}S2 ]I,nb JUSWIsa sIMasIyeuUOSIessp sop gIIofew spueIs ey] 
‘juepusdad ‘aoods 9] suep Jotuuosies aod un.p uoTeULIOY 
B] & JWaNqLyUOD ‘a9AIaSqO SLI9S eR] SUep ‘Inb souguIOUsYd 
S23] aMIp-e-}so,9 ‘oxy onbipousd jayjo yno} jo aummuuoynis 
UOU DIJeUUOSTES ke] IOUTUN[D JUOIeIAOP UONesI[eUUOSTesop 
ap san soy ‘unui ny “UoIssnosIp oun mod (6661) 
Aajpury 3 dnynog MOA — syenbgpeul juos sioruuostes son[y 
sa] anb Jaijiusis ynod agst[euUOsTessp oU9s oun suUep SoJQIU 
-uostes saouonbay sop e xneqoods soid ap aouesoid eT] 


a||ENpises ay|euUOSIes e| Nod anbij@wWeed UOU }S9} UN : UBIOH }e ACu|QOW ZL 


“Npa LNOssMU@suejoy : JSLINCD “0019-1179 


‘OW ‘eIquINJOD ‘TRH Ysnge[ppIW pl ‘eIquINjoD-uNossIP Jo AyIsI9ATU_) ‘soNsHeIg Jo JUoWUedeg ‘URIOH NOdg ‘Ao3'snsus0@)AoI[90OuI's "Ja yony 
:JPLUNOD ‘OOI6-EEZOT “I'A ‘UOurYseA, ‘PEOY [ITH JANIS 0O9P “NeoNg snsuZy ‘Ss’ ‘UOISIAIG YorRasaYy JeonsHEIg ‘Ao_oW JOAN] “| 


‘(p861 
JOUN]IH{ 19 [JOG Suep UOTSsNdsIp Pv] Issne IIOA) sonbruouo0s9 
saguuop sa] Inod ayenbopeur yUSWIAT[ONjIGeY \So gII]eUUOSTeS 
eR] Jastfapowl mod juawanes sax sonbipousd suonouoy 
ap uonesi[nn.| (8661) YaYD 19 ONO ‘Ted Tosuoyy “Ao[pury 
ausI[NOS UO], sUIWOD ‘onbinswered uou no d[9poul 
un Ins s9puoj Os aT]a,nb ‘uonestjeuuostessp op sinpgdoid 
91NO} op dOvdYJa,] JouTULID}ap mod syqejodde onbiysuresred 
uou oNsouseIp uN juIOd ne suoyoW sno ‘onbinsurered uou 
no (2[apoul uN IMs d9puoj aMp-e-}S9,9) onbiysuresred oysoid 
-de aun uojas ssanbijdde ang juaanod januanboy surewop 
np sapoyjou say onb suojoN ‘orjqnd np uontsodsip ey] e simu 
snpuedsi snjd soy uonesijeuuOsIesop op soumueisoid xnap 
Sa] UOs Inb “(pOOT OTJoIOdeD J9 TTeAeIe) SLVAS-OWVUL 
ap onred Woy 19 (3661 UOYD 19 ONC Teg ‘TesuoW “Ae|pury) 
VWIUV-ZI-X suep sasXoydwio jueuayurew yuos Januonbsy 
ouIeUIOp 3] INS sodoxe SOpoujsU SIT ‘(PO6[ IAOPION) 
anoods a] suep Jotuuosies sid un sulow ne.p souasaid 
e] Jed sgsuajoevseo ‘uepied JUOWDT[aULIOFUT 4sa dyITeUUOSTeS 
eR] Wey uq ‘onbipouied isenb yuawayoduwio0d un JasIyenstA 
op jaimeu jno} usAow! un ayyjo ayjo,nb sored ‘onpueda 
SON) 1Sd dIyeuUOSTeS RT JosATeUe 1d I9}99}9p Inod Januonbay 
ouTeWIOp 3 Ins saseq sYydoidde,T “1oMjda}J9,] Inod oanbisoy 
Jano un ysa anoods af 39 yURyoduN onbnyeid suwaqold 
un osod d][anpisal ayyeuUOsSTes eT Op 39 I]TeUUOSTeS Fe] Op 
uoTd9}9p PI ‘satelapay sonbystje}s sap sed oj sueq ‘([onsuow 
}so oseUUOT[HULYOS,p a]JeAIo}UT.| onb yuesoddns usa) 9/ug 
19 9/UG ‘9/Up ‘9/UE “9/UZ “Q/U SaIgTUUOSTes saDUaNbay 
xne JudAINs jusAnod imb ‘siaruuostes syip sord xne 
SUOSSOIQUI SNOU snou “UdUTESTO9Id sn{g “UONesI;euUOSTesap 
eR] e& uoneoydde, ms axe jso gpome juasaid a] 
‘sdua} np sinod ne yUaNTOAD 
sonbipouisd syoodse say ‘soxty ono,p nay ne ‘suoneorjdde 


sou suep ‘([861) Aapsoug suep os9g]NdSIP 3sa Sox 
S}QJJo Sap uoNd}9p ke] op wRNeN ame] Nuepuoge,T 
‘oMuyUuT Imayney op xnenoods soid sop e juopuodsau0s 
‘seo xnap sa] suep ‘Inb (p'¢ UONIIS PT & SVd 9d B SUOISUD}XO 
souleyiad9 §=«suoisse}. snou onbionb) soameuuonejs uoU 
sonbipouisd sauawiousyd sap ay[a9 Tu (Sa\sTUTULIa}Op) Sox 
sanbipousd. syajja sap uonda}9p ev] sed jsa,u uonednoso91d 
gjediouud anou ‘uonbgsuod seg ‘omeuuonrys snssoooid 
un suep sonbnseysojs sonbipouisd sjaya sap e suop 
yUapuodsauod jo aIUIy INayney ouN yO sold say “UassoI9}UT 
snou inb suoneoydde sap axa}Uu00 a sueq “uo, snd 
agjuasaid sa asiooid snyd uonuysp oun { sa\daIIpP SOUTSIOA 
SOS dP d][99 B aNdLIodns jsa aje.noods dsseul PR] JOP ayenoods 
dISUdp BP] 9p UOIZoI UN sUOpUda}Ud snou ‘TeNdOds od Jeg 
“xneoods said ap souasoid 
RP] & JQIsuas }sa ‘seo anne] 19 UN,] Sup ‘Inb ayoR) ‘TeUusIS 
3] Moagid no amenxa MoyNoA yuoreunod sajys~yaQWIOUOD9 
so] onb sure sInaiussul say ‘juouIa\daUIONUT gIyIN0ds 
elas sajyuepuodsauios sayloipouiad sap ajduroo sed jueud} 
du snssasoid np onbrueudp ev] ap ajapowl yno} ‘agououoid 
JUIUWILUVSTJNS Sd ‘SOUISIOA SIMITVA XNe JUIWIIALIOI MIBIE] 
BS Jo INajney es Jed sanjeag ‘oid np doJo}F ek] IS ‘a}UROdUIT 
JUDWIDATILIII ISO anbyseyoo}s snssado1d np syeqo]s soueLIeA 
e] & UONNG QUOD ek] JUOp saousnbay op sues ouN.p 
JHeotput yso oid uP, ‘onbyseyso}s snssaooid np onbrweudp 
eR] IMs sdoUaNTUI sof ouUN ddI0x9 sOUOUOId JeNoads 
pnoou un Jeo ‘soid sao ap UOTdI}Op LV] B JUDSSOIDJUT,S SINayNe 
xnoIquiou op ‘daLMQWIOU0DS | op jo aIUds Np souleuOp 
So] sueq ‘a[qetAno mof op jaJja un no g]euUOSTIes ey onb 
[21 ‘onbrpouisd yuausyodui0d ap sdIpul uN jsa omeUUOTe}S 
snssad0id un.p anoads a suep oid un.p aouasgid ey 


uononpo.wUl “| 


‘a[ejoods 9yIsuap { UONesI[euUOsIEsap ‘ dI1SUap ek] op onbiysuIesed uoU UONeUII\sSd { sajdyjnul s}say: SJID SOW 


‘(4dOO) sonbiuouos9 yuawiaddojaagp ap jo uoIeJodoo9 ap uoesTUezI_, | 
ap 19 neoIng snsusd “S'Q Np jueUudAoId soguUOp sap IMs jULLOd aI[9Y99 opueIS e sed ap opme oUN.p jo UOTe[NUIIS 
9un .p usAoul ne onbijsouseIp }soj 9] SUOIPNID SNON ‘SogsI[eUUOSIeSap S99UUOP So] SURP I]]ONpIsel dulyeuUOSIes dUN,p sdUAaSoId 
RB] 19}09}9p B IIAJOS Issne ynod oI][q “‘oLI9s ouN.p oMeuTWN[sId osAjeue,| JUeINP 9]N ong OUOp ynod yo onbisuIeIed uOU 
DIQIULU IP 99IOGR[Z SO SPOYJOUW! SON ‘SogdIISe 9}1X9AUOD INI] Op 39 ajUad Ing] op UOENyeAg,| Ins yuatIndde,s nb aouasgid 
Ina] ap UOda}9p ap }S9} uN jo xneQoods sold sao ap sinsou suN suosodold snoN “JolUUOSIes j9}J9,| ONb sja} ‘sonbyseysoys 
sanbipousd souguiousyd op douaysixa.| a[eusis aieuuole}s snssaooid un,p aydeds a] suep soid op souasgid eq 


pUINSIY 


_ UBJOH 309g 30 AOAPAIA] A9WIN TL, 


I[[ONpIsoa 9PVUUOSTVS vB] ANOd onbiyIWIeIed UOU 389} UG 


LL 


enBojeyeo ne x-100-ZL .N ‘epeued anbysyeis 
06-12 ‘dd ‘| .N ‘SE “IOA 
600Z uN! ‘ayanbue,p sanbiuyos 4, 


ango[eje9 Ne X-100-ZI .N ‘epeued onbysneis 


d 
CW ,-x)4 
C 
rea (9-943 
Cy |Z) 
‘Iosuuds ‘ 
> YIOR MON ‘aouasafut pooysynjg sof sjool (9661) “WW “‘iauuey PU OHO SCM, <1 MORROW UH 
“L8t-v9P ‘09 I=) I=5 Iss 
‘gq sougs ‘pdyyung ‘suoneindodiadns jAO-AAN 10} Sueoul eare ice ete og os 1G Ke = (“q- “q)q“o r€ 
[290] JO UoNLUII)se soheg [eoLdug (3661) WW “YSoYy 39 “g ‘IexIeEg q ; 7 
“LLI-E91 “$8 ‘uounisossp poousynig SUOAP SnoU ‘OU0G 
uDILauUp ay] fO JDUANOL “SIOYEUISA eoTe |[eWIS JO JoLIa parenbs 
uRoU st) JO UONBUINS? SY] (0661) “WN'f ‘ORY 19 “NON ‘peseld tr tel 
‘6ZS-SI1S ‘I | ‘sousunig fo sjouup ‘suonouny soueLTeA (Z1W) Mx Sx “Hx ((g) “we — 1)(g) “u((q) “uz - 1) Ka <- = 
onempenb yim sarjruey jenusuodxe yeimen (€861) “OD ‘SLOW one 
‘08-S9 ‘OI ‘sousuomis fo sppuup ‘suooury soUeLIeA =f |=! g Q_ 
oneipenb yim sarjrurey jejusuodxe yeimeN (7861) “OD ‘SLOW My hy ((q) “wa — 1)(q) “wu — a ey 
‘TSI-LEI ‘TZ ‘aouadasafuy pup suiuuvjg 
poosysynjg fo jousnor ‘(UOISSNOSIP 99Ae) UOTeUINSA PoOOUl[ay]] Axia 
-Isenb Jo uoisua}xe uy (6861) “AW ‘uosdwouy 319 “g'A ‘oquiepon G2 92 lz ge 
LIS-LSb (9) 1, 
‘QQUaIdg JDINSIDIS “(UOISSNISIP daAe) ssduonbes ajdyjnwW B3uisn 
UOYLINUINS SALI} WOY VUDIOJUI (7661) ‘Cl ‘UIqny 30 “YY ‘UeWEH enb suosod ‘ayno uq 
‘DUO/MPH 7 uevudeyD ‘sisdpouy ies 
ping unisadng “(p007) “Cd “uIgny 38 “H “Wars “f “UID “Vy “ueW]aH 92 42 “'q — '9)(‘q— “q) |aod 
"TRT-ELT ‘€6 “UOLIDIIOSSP JDIIJSIDIS uDIIaUp (q) ‘1 7? ss % 


ay} fO JDUANOL “UOVUII]SS Bae [JEWS 10} s[aPOU JesUT] poziyejousyH 


(8661) ‘da WED 12 AML ‘pnons “y ‘uefereeN “Wl “YysoyH “OUIQUI 9] 
Z11-S6 ‘16 ‘Pyuaworg “s\y8iam AaAmns “4 sap ajuepuadgput ajueysuoo oun ysa “go /(4) ‘10 
pure s[apoul uolsuTy soueLeA oyeIpenb Ajrwe} [eusuOdxo [emjeu onbsind ‘9 = [ “ge /(4)-Le (4) "1(9) nsO]A0Q onb suojoN 
uo poseq UONeUIIsS vore-[[eWIS (F007) “L “HIE 19 “W ‘ysoyD 
SLT-8b7 ‘OE ‘A 'qe'9e EAeNg 
sours ‘AjaI00g poousunIg joXoy ays fo pousnor “(uoIssnosIp 92A®) (I'v) eet Og a [ae KCC 
sjenpisol Jo uoynqmnsip jeisued W (3961) LA ‘THUS 3° “Wd ‘xoD pd 
‘OEI “Z ‘SousuHIg yI]DaH] puD jrI1A “yO0T-S661 s [=n I=5 
‘AVAING MOIIAIDU] Wed [PuOYeN oj JO} UOVeUINSS pue USISOq “(q) "1(4) ns |AOD “<<+ 
(0002) “TA ‘Suosreg 19 “TD ‘AuevoW “yz ] ‘e00y “T's ‘ueunog Oe Le ad 
a1yde.isonqi _ 
rece ih “0('9- ‘Qa<-=(,,2@)0+ 
d 
‘yMsua.s sUIgI09I9 2] ‘(¥)°C = “u anbsing go"qo Inv es 7 
“(9 — 'Q)C9- °Q) |A0D st 
(“yw ,-x)4 ee Q Kae I 
C 9? Ye cee € 
(yO + -<—=(q- )d) a 
(Gaye) | S75 @-OF (aps) ‘QCa- "9 nhc 
(‘y,-x)4 “gO ‘go l=s 
4C9- "Od | <= 
n0.d (q)‘1e nicig poo Lf ca) 3 Ke 
(CY) = “W no (9) ‘Da =0 


saulewop syed unod senbuidwa seuuaiseheg ya sanbiyosesgiy SuONeWNSy : SUOSIed }@ YOZJEY ‘WEY ‘eyUIS ‘WIy ‘YsoYUD QJ 


anZojeyeo ne X-100-ZI1 .N ‘epeued onbysneis 


‘(9961) [JOUS 19 XO_ JURAINS Ud 39 SadURIEdso so] JURUId UY 


'G2°92 ba ia 
T SF 


: Vi Ta aN SG 
(0+ Gye! QCq DKK 


Ss 


ae 


rates (ERE @) I= Qii=0 


dIPIO SUIOTXNAp op 
IojAe, op atlas ua yuswaddoyjaaap a] Jed suosusurt09 snON 
‘yMs [I oUTWIOD suopgdoid snou ‘(q—q)q JOANOy sp uy 


11092 


(CWO + Nat ices ee oil 
(q) "uo 


a-p|a5+ 


(9-9)4 Raa = [(q)‘w — (g) “wl 
PF | giao 


suOAe Snou ‘sadeja xnap ud IO[AR 
ap ouiss ua juawoddojaagp sed nesanou op ‘ueUsUTeyY| 


L(g) “wu — (q) “wlg— 
[(q) “ue — (g) “wah ((q) “mez — 1) + ((q) “we — 1)(9) “we = 
[((g) “we — (q) “ue + (q) “ue — 1)((q) “a — (g) “ut + (Q) “9 = 


[((g) "w — 1)(9) “wg 


USpI,| op suoyred snon 

(8961) 

[]2US 19 XOD op suomdsut snou snou ‘ejI0 Mog ‘se}IeYyNos 
uoneurxoidde | muajqo nod (q—q)q7 sieiq np ajueprud 
uoneunxoidde oun,p uloseq suoae snou ‘ou0g ‘(no 
QIP1O,p suLIa} a] suoJoOUSI snou ‘((g)“w—])(q)"w sed 
juawiajduns ((g) “wu — 1)(q) “a suoumsa snou 1s Guepuadas 


‘(, YO eps] eB nbsnf yoexe ysa nb (.(¥ + P/ -¥ =) 


I=! 
(Ol'v) xcp'u = 1)(g) “wen 19. 


cgi NOM <} 


L 


red suntxoidde 
1S9 (Z'g) Op SIQUIDUT OWATXNap Np sULI9} sUIeTXNap 9g] ‘NO,q 
(90 =(.% ® (yO +9 =(,,%0+9=9 
anb Jajou ed suosuswmu0d snoN “7 sWe10sy) Np sAnaig 


(6°V) 19 (ZV) ap anoogp surgI09ty 9] ‘asatpodAy 
aqou ep nyea ue (¥)"O = “uw 1a “(,_ ¥)O =(g),-K anbsing 


69 


(et0+] a “w — 1)(q) "ui <|o.« 


(6'V) 
[=f a 
Ec "w — 1g) “un <(e om 
I 'u z\ 
f f A 
(WO a x(q), 'x((q)“w-1) 
; ‘us f#4ls| 
(q):“ui((q) “ua — 1)(q) “uartnon OE + 
apt pay’ bee OD 
“x(q),- [x (((q)"u — nneutneg | = 
y z 
I=f 
1 fi 1 
ce = 44d)" (2 
z 'y 
® QUSW SNOU vI9—) 
(3'v) (,M0+ 


x(q), x(q)“ — 1)(q) “a((q) “a — 1)(q) "ue = 

[((q) “ue — (g)“ue)((q) “ua — (q) “ua 
re) (“0 = I(q- q) 'x\q aonb suojou ‘owiguLaq 

(ue¥*)O = 

(a — 9) ix 1a ((q) “uz - 1) = 

[“x (4-9-9) (x]q((q) “wz - 1) = 
(9-9) (x"x (9 - 9)((q) “wz - DG - 9) (XIE 
onbstnd' 

(240 = 
(9-9) (xx (9-9)((q) “wz - (9-9) [x1E 
931] 3OU SUID} 9] onb suousTjNoS 


- ‘asotodAy 
ajou op nea usa (“u)Q spo] e.nbsnf 3}991109 ys mb 


(Lv) “x(q),-& 1X ((q) “me — Dg) fu = 1g)“ - (g) "ag 


uonewrxoidde | suouajqo snou ‘(¢*V) 19 (py) sed Suour 
“urenbgsto) (“4 70+ (9) 1 px"x) a = (9-9)( IX) 
SUOAB Snou ($8) suep ‘onno uq -( lO a (q) arg 
=[,4-949-Q1a 20. (MZ = (MDA (Qe -D 
(q)"w =(“4) 4 onbsng (,¥)°0 + (9).L,-% =9- q ‘ou0q 


6002 unl ‘ayanbusa,p sanbiuyoe | 


(9'V) (q)Z-= 


X(W-DW,X-= 


I=f =! 


1X ((q) "wm — Yq) “we X= 


ts eo |S - tua 


no (/4)“0+(4-9) [(@MLA]+@ML=()1=0 

‘adeja oun ua iojAey op douiss ua jyuowladdojaasp seg 
0= (QL 
SLRS 
Wns |! 

suUIOS suopssoid snou ‘[ AG qVq - q)] q IANO op uy 


ficy i=l 
ay 


enb _ayios ep. 'x((¢)"w—- 


(cv)  [,@-9)(9-9) 4 ix'x]4 = 


[(q-9) ix"x (q- Mla = Lq-9 ixl7 


anb suoArasqgo 
G 0.9) “Q. 182 981[3au sua) sonal 
aq ‘nod (,,-1°O = ll¢-4ll ‘uod (_¥)’0=(9),-% 
anb ainosap |! ‘(¥)‘O=(g)z_anb asanody, snog 
"| SUIQIO9Y} 9] JURAR TUIJSpP IS2_(G)X NO “((g),-< ‘O)N TO] op 
yuouanbyojdurAse so g—q aonb gnooap TI “(g661) YysouyH 
yo Tepes oq “(19-9 )) 10 4So o3I[39uU sUIA} JoTWoId a] 


snou “ayinsuq, 


(pv) ° C — 4) ix"x (4-4) 


(4 


(gy'ue- N+ (4-9) lz A(q)'u = 1(q) = 


(q— 9) ix"x((q) "wz — 1)((q) “uw - Dg)" 


rg 
4-D—+ 4-9) x (@)'u- Yi)" lz = 


L(g) “ue - (q) “wg 


anb ymsua,s qt ‘ix "x((q) “ue 


~ 1) (q)“w((q) “wz — 1) = (,9090)/ ((q) “ut ,0) van yUR}ON 
19292 


sig ck noun 2g Shea 
(9-9) @yiu.e 14 his, q) 


in <atkiain Jr: (g) "a 
A@ we 


ongo[ejed ne X-[00-Z1 ,N ‘epeued onbnsneis 


‘sadej9 xnop ua Jo]Ae | ap atias ud yuouraddoyaagp seg 


(¢-v) | “a —(q)“u) 


‘us fels| 


((q)“u ~(q) ‘ui “a'n Zt 


Mayu (g)'uyin Z lz a - 
6 


'u 


= 
aya —(o'uy's <3 uals 
g t . 


ed ~ aad)" Is 
iG 
‘suoynoyeo snou “uIjuq 


TR 


(z’v) “((g) “we — 1)(q) “uatn — 


=f 
sont 


anb aplos ap 


(4 


ny) 4 
((q) “uw — 1)(q) “wy 


[ I+¥ | i+) 
((q)“u—1)(q)"u) 2X 


ae f+) _ 
I 


[ww “ut — 1)(q) "we 


+('d-1"d)g 


pie Wee 
Nd 
0+ ( TA 


Al At) | 
: el es 
“Al mee 
NC 


71+) | 
aa 


(“d—(q)"u)(“d—"a)q* 


+ de 4) 


1+) 
a 


c 


{ aa me (q)"u\ 3 @ 
[(/a (at 


+ + 
4 —()fw — a oat la 


Lavin Ane X 
+('d — "4 = 
7 ( eager le 


(4 


(6 


Ad - 40)a 


saulewop sj}ijed unod sanbiidwa sauuaisaheg ja Sanbiyosesaiy SUONEWISY : SUOSIE, }9 JOZ}EY “WEY “EYUIS ‘UY ‘YSOYD gg 


anZoyeieo ne X-[00-Z1 .N ‘epeueo onbnsneis 


suojnoyeo snou ‘ayinsuq 


{=f 
yikes faygtn'< - [cata a 
SIV 


(I'v) 


(o-208p 


c 


(S18 


et 
Co <2- (“Woda 
(6 


‘nO. 
=f 
a cicta-fay'a’< jal d= eZ) fa 


=f =f 
[ea fan's <q(Cemea: x| q 


id = (4 |"d)qJ anb yuejou ug 


[=f 
: fi fi zy hi fi fi 
re (Cr) “| ae + 
fig — layin’ ig fda ia 
( ye A+ ue = yin < A= 
G Z u 


fi fi ft a fi 
(capesse-got’- 


c 


=f 
cd Teneo <3 = (“d- iwg = 
c 


(“at WOd 


aay 
“[ WIdIODU} Np dAnalg 


axouuy 


“UONUDAIIG PUL [ONUOD aseasiq JO} sIayuad 
sap sonsneig yeoH JoJ Jaye jeuoneN np suormdo 


LQ 


Sd] JUoWMessaoou sed jUd}ITJoI dU 39 SINaINe Sap saT[Io 
JUOS a]OILIe JUasaid Np suOISN[DUOD sa] 19 SUONR}e}SUOD SY] 
‘« SoPEUINSY UIBUIOPANS poseg-[Spoy » s[MHU C87T98CC8C 
orgumu jafoid a] snos DQD/SHON $2 Jed juowr[onred 
sgoueulyy 939 yUoWaTeS9 yUO xneAe SIT YSN XI op 
P8IZILO-SAS asinog ev] Jed ored ua sooueUly 939 UO Moyne 
augienb np xna) ASN XI 9P 9ZPIEIO-SAS 1° 68SLIEC0 
-SqS sasinoq say sed oied ua soouvuly 939 JUO Mone 
Joruaid np dyd1Eyool Op xnNvaed} SY] “ape | op samougjue 
soyonegs,p jalns ne sjoMysuod sare}USUTUIOD SIMIZ] Ip SIN3} 
-euTUIeXd xnap jo JUIOlpe INd}depaI 9] SUOTOIOUIOI SNON 


SJUIWIIII AIUD Y 


“SaTJa1oyyo sanbiytjod ap uorsnyyip ey & 
SOANLIAI Sd][IUUONNNSUI SouULIOU Sap jdadsal Np 19 SapoyoU 
sop Ipuojoidde snjd uawexs un jelaIssaogu sonbrytjod 
ap UONLIOGR]I,Pp }X9}U09 UN SUP SpOUJOU 9199 op UOTIRd 
-]dde J Jueysnyt 30 e.nb tor saqyuasaid yUos ou sYydIOYIOI 
ap xneaey sao onb JausINos op judIAUOD [I “UyUq 
‘loa}sod e saanoipoid d smayea sap uahouwl ne uatsaheq 
onbryorels1y ajapoul np uoyenbeape.| 9189} SUOAR snou ‘ney 
snjd suUOnUSU SUOAR | SNOU sUTWOD ‘ano Uq ‘oUTeWIOp 
np onb isure 9y1uN | op NedalU ne ‘[eUly s[Qpou 9] SUBP JUDUT 
-onbyewioyne o9139}UI So BdDUepUadap PY] ‘sdURLIVA ap soquRs 
-odwi0d xne jd UOISSAIZAI Op SJUdIOIJJI09 xne sUOTNQU)sSIp 
sop Yuoumuoenbasqns ‘a samojege sjajyo xne suoTNqLysIp 
Sop osnquye suoae snou anb sioj ou) “somoyeagye sjayya so] 
}9 SdTQeLUIeAOD say JURYSeS ‘JURpuddapuUT JUDAT[aUUOTIpUOD 
anb jsa,u onbyatjodAy sjopow! oanou “uepusdad ‘saseusul 
Sop MILUDUI.| @ 979109 jsd dOUBINSse oUN Jed oMpIIANOD 
eR] ‘uouonbuiosaye9 san ‘aonb ja oseusu-ajgnbua oun 
}So ayeuIsuO ajonbua,| onbstnd ‘sour;quiasiesA ey op [Nye J] 
suep saseuaul sop soddeis ua ast ey op a3dur09 sed yuo] ou 
asdyeue ajuasoid ey] “INd}eUuTUTeXd UN dUBI[NOS &,] SUTWIOD 
“SOOLIBAT NUL Jd 
SOOLIBAIQ SOITRUIG SoQUUOP ap asAyeUR,] B S}EI[NSOI syuasoid 
SO] dIpua}9 SUOTNOA snou ‘IaTNOWIed u_ ‘alley e jUD\SdI 
xnvaey) op dnoonvsq ‘sutouuvaNY “WOLa}sod e sanoIpoid 
dmogea ve] op udXoul ne sJepoul ayou ap jusuajsnle | op 
uonenbepe.| op }UdUaTeS9 SUOINOSIP SNON ‘sonbrydeisowlsp 
sonbysuajoered sane .p 39 dxas np ‘a8e,] dP JUDUIISIOIO 
red agsiaipgns ‘onbyeise uonejndod-snos ey mod sonbryo 
-Ieloty 3a sonbuidwa sauuarsaXkeq sourewop sjyjod mod 
SUO}PUITSS Sap SUOSSTUINOJ 9 JJURLIOdUT sa UONSaNb ay29 
JoplOge & IOOUSWIUIOD Sp SUOUDSA SNON ‘s}e}q Sop 39 XNeIOpoy 
soulstues10 sap sanbrtyod sap uoNRIOgeIA,| Ins J0uaprloul 
oun OAR .p a[qudaosns ‘jueyoduit say swieyqoid un yUsWI9A 
-I}UIJap jso ‘solreyouTW sodnos soy tured yuowarotpnorped 
‘sognsse uou souuosied op uotodold ey] op uoneUINss 7 


UOISSNISIP }9 XNVABY SANJNY “QUINSIY * 


600Z uIn[ ‘a}anbue,p senbiuyoo) 


"saouuop xne 
gisnfe usIq yUoWs;qeuUOSTeI! ang s;QuIas ssodoid sjapour 
3] DU0q ‘“gPg¢e‘O 389 aUepuodsauios d maeA ey ‘USUI 
-aanoodsal ‘¢¢6L¢ Z 18 09‘7Z JOS DIG 10 Yd ap saaumsa 
SMgVA So] 39 ‘ssoguUOp op jUsUIOdnoIs op sed e A.u ]I.nb 
ayiusis mb 99 “(amojeaye jOYJo UNONe sues JULINOD s[apout) 
gjduns onbysisoj uoIssarsa1 ap s[apow a] Issne suojsni{e 
snou ‘suoq ‘jnoipeid moanod moayjiow af yueAe s]apoul 
3] Mstoyds mod ja sjapow np juswiaysnfe,p soiojI9 sUTUIOD 
ssesyyn yuos aIq 39 Yd smayea say uawaTjamiqeyy 
Trbly 7=O1d 3 SL‘9¢= 7d “qH ashyeue anou mog 
(007) 1109 39 UeUTaH 
NOA ‘sjlejap sap Mog ‘uoUs\sod e uONRINUNS ey JURSTTHN 
Ua a9UIISa souRIASP kT ISA ((,Q°¢)G Je Saneuiered sap 
SagUISe SIMa[eA xn ag[No]ed souRIAgP BT IS2 (Q ‘C)G NO 


angojeeo ne X-100-Z1 oN ‘epeued onbusneig 


(0 “4g -(()0 “Oz = 1d 


JULIO} BP] SNOS 9UIT}SO SO OIG IJ ‘UoUd ze 10] 93N0} B a1D0sseV 
}S9 [IS Q Mova ev] Jo (SOOUUOP Sop UONNQU)sIp) [euUIsLIO 
gJepour np onsred jez [I.s | mojea ey] puaid anourered 
uN NO ‘o]epoul J] SUBP s}UTeQUOD UOU sanouTeIed ap siquioU 
3] SWIWOD ona ana yned Td mayea ey ‘aaumysa sanorpaid 
OURIASP PI NO (UOlaj1ud UoyoUof{ul aouDiaap mod 
JIC) 22uniaap ap uo1OULsOful, Pp AABJIAD J] 32 Z/(QOURIADP)IVA 
=7%d moaea ke] g[nojeo yuoulayesa suoae snon 

US sed aor hae dence | | 
suep S}e][NSoI SUOG dp JoUUOp [quads sesodoid gH spoyjoul 
RB] ‘ouOGg “WoUIa\snfe UOg UN sWWOD s9}Te] JUSWSTTaMIGQeY 
so ¢€{(9 B& amouiadns dinajeaA ou) ‘soTquiquias smoayeA 
Sop juesinpoid (p‘o) ap x1oyo sayne saz (Z0‘0 ‘Z0‘0) 
=(p‘2) mod g[7p‘0 1sd s9umjso dimojea ke] ‘Uatsakeq 
onbryoreisty onbysiso] uolssaiso1 ap s]apour a] nog 


SUOHL1I YOO [ 9A" BUNDY) ‘sauIey) ¢ ‘« SNq‘Papoul UvISE » & Song IEpPoy] | BNsIy 


00E C 
! CE C SoURIASP 
| OVE C 


Of BE BE FE CE OE BZ OF be 22 2 Gs Gb wl Zi ClBRIVE KET 90 


i I : i we f 80 d, 
sit caanedsonattetet?sotyttsat 


BUIBIS 


Gil 
pI  ¢eydye 
91 


oe 


60°0- 

0 

soo 8 zeqdye 
Loan 


S0°0- 
0 jeydye 
S00 


one sen 


I orydye 
S‘I 


% OX OP S2|[VAIIjUI Jo SouRIPSUL 


: 
: 
| 


soedsa .p aynej anbuoy nesjqe} , 


a Cc I 0 (I 

8 z Zr 

® = iP 

. ee OY 

@ = 6E 

e 3 gE 

e E ae 

@ mm oF 

2 - Se 

@ = re 

® z So 

° = Ze 

Ye e Le 

be : O€ 

a bd 62 

. . 8¢ 

@ = ae 

® & 92 

° = SZ 

e E $2 

bs ‘S £¢. 

@ = Lo 

. = a4 

a = 0Z 

e 5 61 

e 2 QL 

® = vay 

° z Sl 

s = S L 

2 = vL 

e & El 

8 é ZL 

° : bb 

e ea L 

° a 6 

e = 8 

e = 1 

° = 9 

@ = S 

® = $ 

® = ¢ 

® = Z 

e = t}d- 
~! = BUIBIS 
e = ¢eydye 
® z zeydye 
e 3 peudye 
e Bay oeydye 

+O a Sali Il ¢ I 0 hs 


aureys anbeyo 
Inod % 0 ap 2]]BAI9}U] 


neadeyo-y 


saulewop s}ijed unod senbuidwe seuuaiseeg ya Sanbiyosesaiy SUONEWNSY : SUOSIE, }9 HOZ}JEY ‘IEW “BYUIS “WIy ‘YSOYD 99 


andojeieo ne X-[00-ZI .N ‘epeueD onbnsneis 


‘sajaudoid sma] sIpnys 39 BdUSSIOAIP Op SaINsoul 
ap siquiou ureyi99 un ssodoud yuo (8661) YSOYH 32 puejyjeyH 
‘Pp ap sajqissod x1oyo sanne.p ajstxa [I “‘Wourajsazrueyl 


ikgid = Dod aida 

( (phen LA eaten Geyrseoen (nF (yOP 
(7) as ul 

(4d - Dvd [=f 1S! 


ue des rae a (nd ‘A)p 


}S9 p DDUdBIOAIP Op 
a]qeue ey ‘sIopy ‘uoneIEN 47 ej ap ede “d sop sagiguad 
SINICA S23] JUOS ((jo)dx0 +1) /((,9)dxe = = ea no (fd 

=) jd VG id - 8)) oy ras dere sop 9d 
=) gee suoanod snou ‘((“9)dxa + )/(“9)dxe = “d 
=(“d\"x)q onb juejoN “JoMONIed ua oun dieprIsuoo 
SUOAB Ud snou ‘adno90 snou Inb sed oJ sued “p dUdSIDAIpP 
op g[qeuea ke] Imod sajqissod yuos x1oyd simotsn{d 
‘9390188 UOISIOdsIp op SUTWILISeIP J] SUP IDINdDSqO 
ane weunod ‘uowenne ‘nb ajapour np janjusAd [ed0] DeY9 
un JasiyensiA op jenjouued nb a0 ‘sadnoss-snos sIdAIp 
Inod soummeiseip syuarayjip juestjyn ua onbrydess asAyeue 
aun Jajndaxa juoulayesa suoAnod snou 4uoWaT[oIMeEN 
“gHIOUI ouN,_p nad eIayV99S doUIT}Sa d INaTeA Py ‘SUOTTHURYDE 
spuvis so] Inod 41p yuswIayNY “Uog 3sd JUSUIA}SNIP | “19-a]]99 
snos ‘aylour anne] 39 Soisap Sp & d}IOIP ke] Op Snssop-ne 
JUSANO] os uOIsIOdsIp op sUTWIeISeIP Np syutod sap uoAUS 
SNIOU BT IS “a]Jayo9 owgUI BT MS ((2°(,4)p seanorpgud 
sIng[eA sop uoNoUoJ Us ((3‘4)p Sagst[eal smMo[eA sop 
UOISIadsIp op SUIWIeISeIP UN JOsI{yN suoANod snou ‘s;apoul 
np yuowiajsnle | op ayyenb ey Ja1~U9A Nog ‘aTJaMjIQey soLneO 
-Ipur uoouoy eT sa 7 no WP? PSF WPL He yw aeBo 
}sd (UoUa\sod e& aanoipoid d mae sajadde yuawayjanjiqey) 
soumysa d moea ke] ‘OUDG ‘a[qeUeA ay90 ap dsesi]eal 
INIA BP] SPIOXS dIINUIIS 9dUdBIDAIP op a[qeLIeA eI syanbsoy] 
suep seo op uomodoid ey sed d mayea oun suowmnsa snou 
‘sind ‘QUUOP I[9POU a] ddA PJODOeSAP Ud JUdUIdIOTIUS JOS 
A SaguUuop Sd] Is SUI9I}X9 INayeA ouUN vine INb ‘(9 ‘4)p suosIp 
‘QQUdBIDAIP op s[qeueA ouN MsIoyd sed yueSudUTUIOD UA 
B[99 SUOSIR} SNoU YUoWIdTTOUUIO., “¢ sadAIasqo saguUOp xne 
SOILS 919 JUSTeIADP sanbyayjodAy sanbriyda. sao ‘stogid 
JUSWZ|QvUUOSTeI JS2 IQPOUT 2] IS “9[NUIS (2 a] JUYORS 
A ap seen lo] RB] op 9m sa (Y T=) ‘(4 No 
‘ant “7 Qy€ suostp ‘saguuop sap sanbysmjoddy sanbiydg1 
Y yURUSUTeUI sUOTNUIg “OWO/ wouRnuns oun juesTyyN 
ua Wouajsod e 10] aad aP (3° ‘( 3 SUOTTHURYDE soy 
911) suOAR Snou aonb suosoddng ‘4 yueyoers Q ap WOLIa}sod e 
susuop ey “(C|Q)TT 39 2 Jueyoes A ap aysuap eT (Q| 4) / 
‘snuuoourl sojoutered op maj99A a] 9 ‘sagArasqo saauuOp 
ap Ma}d9A a] A POS ‘aUeAINS RI 4Sd [NO[eO ap suas 90 


S92 


ap a[eiguas uoNeoyysnl ey] ‘sosodoid sustsaXheq So[gpoul sap 
uonenbape | JaijU9aA Inod souustssheq d smejea so] g_noyeo 
SUOAB SnoU ‘(pPNOZ) ‘[JOO}2 URW]IH op Ie\sUI.] Y “o[Epoul 
np juourajsn{e p onbueuw un onbrpur sagasesqo soguuOp S3] 
}2 SdgigUas SaguUOPp Sa] ond JURLOdUT pede UP, ‘SddAIOSqO 
sssuuOp xne suoTTQUeY99 sod JareduI0d ap jo soenbrydo1 
saguuOp sep UoLIajsode saAnoipoid 10] ke] op soagfnurs 
SINJ[eA Sop Jo e daJSISUOD SagUUOp xne s[9pouwl uN.p 
yuouajsnfe | ap auyenb ey JsIJU9A op ustIsgheq UsAOUI UP) 
‘(p007) UIgnY 19 
Ways ‘UlpIeD “UeUITaH op cd dxouUR,] B M9}99] 9] SUOAOAUAI 
snou ‘9sI[Qn suOAe snou onb JaIdI3O] Np sgT]IeJap UONdudsep 
aun Mog ‘(aouR;quiasieA BO] Z-) oURIADP ek] 39 SNOUT 
-ered onbeys mod wousjsod & s0udI9JUI | ‘IOIP 9199 J] 30 
sojatjesed soureyo burs sap yuowIayoneAsyo 9] ayUOUT oYyones 
200 J] “[omMsy ey e sgjUasaid jso ayjamred on10s ou 
“VdOQO |ato1soy np uaXfouwl ne sanbrydde 939 yuo simMeID}I] 
eR] suep s}udep sonbysouserp saigjlI9 sanne simoisn[g 
‘sanoueied sap unoeyds mod (a0uasIaAU0d ke] eB [ =) 
9}1UN | ap sydoid say ang UWe[QUIAS aI] 10 Satqissod aT]aY99 Pp 
UOTJONPal op SINIJOeJ SOP IOUIBIOAUOD PT SIJLIZA SUOAR SNON 
“SUOTILIO} YOO [ Op aeuTul adessnuaidde .p apouied oun daAR 
‘000 I a]]!e1 Op ounoeyo ‘souteyd bulo op SIAIas SoUIWIOS 
snou snou ‘Ialjnoured uq (7661) Uqny 19 UeEWTaH ap 
samdsut saumuios snou snou ‘sqqiy ep IMouuoT[HUeYDE | op 
QOUSBIDAUOD OP SONSOUBLIP Soy 39 DIANA Ud OSTU PT Nog 


uaisgAeq onbiysies91y 
3]Q2POUI Np d1AND Uo stu 39 SIQSOUSLIG: °9 


E710 001‘0 ¢L0‘0 Ss00 pueIsy 
Srl‘0 9E1‘0 9710 6110 ¢€ uRISy 
$80°0 790'0 €r0°0 1700 7 ueisy 
€rl‘0 ETI L60°0 €80°0 | ueISy 
E710 €010 080°0 €90°0 [BIOL 
snjd ja sue ¢9 
S61‘0 €07'0 807'0 6170 ponbueisy 
LET‘O Ivl‘0 Lvl‘0 OST‘O ¢ enbyeisy 
rel‘o SE‘0 LEI‘ LEO Zanbueisy 
6S1‘O 0910 0910 Z91‘O | anbueisy 
7910 8910 ZL10 LLI‘O [20L 
SUR $9 B 8I 
SLI‘0 ILT‘0 S9l‘0 S910 panbyeisy 
vIl‘0 vIl‘0 LIV‘0 C110 ¢ anbyeisy 
€80°0 1L0°0 790°0 970'0 Z onbuyeisy 
vil SOl‘o L60°0 L800 | anbueisy 
LEV‘ I€1‘0 9710 O7I‘O [ROL 
sue /[e®0 
(I=wWaa (s‘o=\) aa GH S290 


000Z ue sanbyueisy .p ednos 3] 32 ade .p 
adnois af uOojIs ‘sIpeyew-soueinsse p sed juede.u suonsodoig 


Z neaqe | 


6002 un! ‘a}gnbua,p sanbiuyse] 


“suostereduios 
Sd] SUBP SOUdIZJOI OP JUSANOS JUIAIIS Sa}aIIP suUOT}eUIT}SS 
so] ‘UonesoIse.p neosalu 90 e.nbsind ‘sjuestejsyes zasse 
JUS[QUIIS s}e][NSoI Say] ‘snjd ya sue ¢g sap adnois a] mod 
a9-jnod jnes ‘sajsaNp suoyeUsa sap sayooid zasse yuOS 
sourewop s}yod mod gH 19 qq SUOTeUII)Sa say] ‘UOTeSaIBe _p 
9Ag[2 Snjd nesatu 99 Be nb a19Av,s |] ‘(p onbyeisy) sanbyvisy 
sane so 30 (¢ onbyeisy) sIsy,p sudIpuy say “(Zz enbyeisy) 
suiddyiyg sa “(y{ onbyeisy) stouryy say mod jusuwisy 
-[onprArpur ‘snyd jo sue ¢g 39 sue HO B QI ‘sue /]| BQ OARS 
e ‘ase.p sadnois sioy sa] suep saginsse uou souuosied 
op suomodoid sap oaieumos un ouuop Z neaqr) oJ 
‘69 ouUTeWOp 2] Inod ayynu smmoauwlap 
geNp uUuoHeWNss |, uepuedadg ‘sa}eNp suoneuUNsa 
xne nod aploou0d 666] & 66] Inod suuaXour ey ‘said 
suondsoxa.p nod san enb aiaae,s |] ‘saj}oolp suoreurysa 
xne nb joynjd suoneumsa sao e& gH 10 gy SuONeUIsa 


€S0°0 0S0°0 S810 0910 
6+0°0 940°0 £910 Srl‘ 
vb0'0 Zr0'0 1010 060°0 
€90°0 6S0°0 OI1‘O €L0°0 
L100 $100 vbT0 9970 
$100 r100 0910 9S1‘0 
7700 7700 O11‘O ZOI‘O 
€70'0 7700 ZZ 9570 
¢7z0'0 7700 9910 7910 
9700 ~70'0 O8I‘O r81‘0 
rS0'°0 1S0°0 €Z1‘0 L710 
L€0°0 S€0°0 1€Z70 LIZ0 
€60°0 880°0 160°0 090'0 
€S0'0 0S0°0 8010 880°0 
1S0°0 8400 0L0°0 Lv0'0 
690°0 $900 660°0 990'0 
L100 9100 6£7°0 7970 
L100 9100 €c10 S710 
8700 9700 vol‘ 7010 
¢70'°0 €70°0 v870 ZOEO 
¢70'0 €70'0 9€1°0 8Z1‘0 
1Z0°0 0700 9910 091‘0 
¢s0°0 1S0°0 1S0°0 v£0'0 
0€0'0 820°0 670 9670 
S910 S59 A) 9€70 LS1‘0 
780°0 LLO‘O 8r0'0 Z€0°0 
9710 6110 8970 667.0 
1€0°0 6700 9710 971‘0 
re£0°0 Z£0°0 0710 S110 
0€0°0 8700 780°0 €L0°0 
€90°0 650°0 €rl‘0 ZE1‘O 
070'0 610°0 9010 ZOI‘O 
0700 610°0 7810 8810 
L=% S0=% b=U S0=% 
(am 7-9 (qm) 3-9 aa aa 


angojeieo ne X-100-ZI .N ‘epeued onbysnyeis 


Sg] Joreduiod ap onb sure ‘saguue sayne.p mod onbysu 
-9]08I2D SUIQUI RB] JUopessod QNDZ Ud a[NU }jIK}a d}OATIP 
uoneuss | sjonbso] mod saurewiop say Is JoUTUIeXd.p 
ynq mod juowosredtiouud kv 10-3]]9d “6661 & L66I Imod 
SQ}DOIIP SUONRUWT}Sd Sap sue SION ap suUdAOUI eT JURUAJUOD 
suU0CjOD + oun jUsUIa[es9 SsUOUUOp snoN ‘suoneUIsa 
“Snos sop JuoWsTjemIgey Jos ((,_¥)O sua} np ajduros 
sed juouuen ou inb) sadAj-smoua sap saareu gy suoy 
-eUINSO Soy] oJonbey] uojas opryea onbyuo ey] e yUoWsIE] 
puoda1 sasodoid oysoidde J “qq suoneumss sap sasumnsa 
soayeutxoidde souussow sonbyeipenb soa sap sure! 
e] onb Isure ‘gp suoneuMssa xne sagtioosse sadA}-simaua 
Sd] JUSWATeS9 suOjUdsaId snON ‘UOT[HUeYDE,p aT]Ie) aINed 
Re] Jed JUSWIZ]qQeIOARJap Sad}dayJe JOS GH 19 Gz suoTeUMsa 
sa] “atwieap 99 mod ‘onb onIeuuoda1 yne} I] “69 oUTeWOp 
Np wuOIsn[oxa,| e& ‘sauTeuop syjod sIdAIp say Inod % ¢¢ 
B® %Z op yourA sonbneisy sop adnoss a] mod saginsse 
uou souuosiod op uomodoid ev] ap qH suonewmMsa say] 


6£0°0 S710 L800 111‘ vl 96 
700 €Z1°0 8100 801‘0 81 66 
6700 180°0 S100 L90°0 91 v6 
170'0 0£0'0 000°0 0000 ~—s«éO £6 
0720°0 €870 S870 80€0 stl (40) 
1100 0sT‘O L9T‘0 6r1°0 €Sl 16 
L100 $600 9r1‘0 980°0 LS 06 
7700 S970 0170 6870 IL 68 
8100 SS10 a €S1‘0 69 88 
1Z0°0 S810 860°0 7610 CS L8 
9€0°0 LEl‘0 9S50°0 9€1°0 Ol 98 
L700 €61°0 E110 681°0 GS $8 
9¢60°0 8700 190°0 000°0 14 v8 
S00 890°0 L100 6¢0°0 el €8 
700 $700 S700 000°0 Cl C8 
€r0'0 67200 000°0 000°0 ib 18 
07Z0'0 1870 rico 80€0 Sél 08 
€10°0 LIT‘O €91‘0 OIT‘O CTI 6L 
07z0'0 €010 810 860°0 6£ 8L 
$700 Ele 0 770 8Ee0 89 LL 
8100 0z1‘0 all) E110 6S OL 
910°0 OsT‘0 L60°0 6b1°0 €8 SL 
v£0'0 vz0'0 vv0'0 000°0 Ol vl 
970'0 9670 vrl‘0 ILTZ0 Sv €L 
1010 S700 7810 000°0 G CL 
0S0°0 €70'0 160°0 000°0 4 WZ 
860°0 I€€0 000°0 19€0 C OL 
1700 6710 E10 9710 Ske $9 
¢7z0'0 aun) SLZ0 SOLO 8¢ v9 
0700 690°0 €60°0 ¢s0'0 £e £9 
vb0'0 0z1‘0 9110 raun) 8 09 
$100 ZOI‘O SIT‘ +60°0 vL SS 
1€0°0 6610 6910 aval) 81 vs 
6661-L661 
(qH)7-2 4H guuasoy, = -aqDalIq.s ‘us aurewiog 


0007 2QUUE : sginsse UOU sanbyeISy,p suoHs0do1d sap sautewop s}yed 1nod suonewysy 


(ays) | neaqeL 


seulewop sjijed inod senbuidwe sauuaiseheg ja Sanbiyoiessiy SUOHEWSI : SUOSILd }9 JOZJEY ‘WeYy ‘eYyUIS ‘Wy ‘YSOYUD 9 


angojeieo ne X-[00-ZI oN ‘epeued onbnsnes 


€9 


7r0'0 
Z£0°0 
6£0°0 
re0'0 
€80°0 
rs0'0 
7L0°0 
$500 
$60°0 
L710 
7200 
7200 
6200 
L700 
£r0'0 
0r0'0 
8£0°0 
rs0'0 
1L0°0 
190°0 
710 
0z1‘0 
9700 
7200 
L¢0°0 
rc0'0 
L¢0'0 
££0°0 
$900 
$500 
L90°0 
SL0°0 
pL0'0 
$60°0 
€Z0°0 
1Z0°0 
Lv0'0 
S£0°0 
re0'0 
££0°0 
O11‘O 
790'0 
LS0'0 
pL0'0 
780°0 
9100 
pZ0'0 
1Z0'0 
940'0 
€r0'0 
6£0°0 
090°0 
t=% 
(am 7-9 


0r0'0 
0€0'0 
L€0°0 
Z£0°0 
8L0°0 
1S0°0 
890°0 
7S0°0 
060°0 
6110 
07z0°0 
070°0 
L700 
$700 
0r0'0 
L€0°0 
9€0°0 
1S0°0 
L90°0 
8500 
SII‘ 
E11‘ 
7700 
1Z0°O 
S€0°0 
Z€0°0 
S£0'°0 
1€0°0 
190°0 
7S0°0 
€90°0 
1L0°0 
0L0‘0 
680°0 
1700 
6100 
rb0'0 
€£0°0 
Z€0°0 
1€0°0 
0) 0) 
090°0 
vs0'0 
0L0°0 
8L0°0 
7L00 
7700 
6100 
€r0°0 
170'0 
L€0°0 
LS0°0 
S0=% 
(qm) 7-9 


vZ1'0 IZ1'0 
9L1°0 861°0 
990°0 950°0 
030°0 €50°0 
SOr‘0 660°0 
ZO1‘O 890°0 
990'0 rr0'0 
S£0°0 €70°0 
9L0°0 1S0°0 
LEI‘0 6£1°0 
8Z1°0 0710 
O€l'0 €cI°0 
ESTO 9S1'0 
8L0°0 7S0'0 
aaul) rel‘0 
re0'0 €70°0 
€90°0 790°0 
860°0 $90°0 
rIl‘0 Z11‘0 
0L0°0 Lr0'0 
$s0°0 L€0°0 
€Ll'0 ZITO 
8710 S710 
6910 8810 
velo ISO 
$500 L¢0°0 
ril‘0 8010 
vr0'0 6700 
9r0°0 1£0°0 
€60°0 790°0 
Z91'0 8010 
6L0°0 7S0°0 
€r0°0 8Z0°0 
0070 L070 
velo €£1°0 
7900 10'0 
ZS 10 Iv1‘0 
L710 €Z1°0 
7S0°0 S€0°0 
6£0'0 970'0 
8Z1'0 O€l‘0 
SvT0 SL70 
veld 060°0 
E710 780°0 
pSl'0 0910 
v77'0 7a) 
ZITO LOI‘0 
rrl'0 rel'0 
0910 €s1‘0 
ILI‘0 £910 
780°0 9L0°0 
851°0 8r1‘0 
L=y s‘0=% 
qa qa 


7700 
Le0'0 
€70'°0 
0S0'0 
Le0'0 
S00 
re0'0 
9500 
9L0°0 
910°0 
910°0 
1700 
0700 
6700 
8700 
¢70'0 
S€0°0 
Sv0'0 
7r0'0 
€L0°0 
1L0°0 
6100 
9100 
7£0'0 
0€0'0 
$700 
€70°0 
6£0°0 
re0'0 
7r0'0 
0S0°0 
S700 
950°0 
8100 
S100 
0€0'°0 
$Z0°0 
7700 
€70'°0 
790'0 
S700 
950°0 
Lr0'0 
1$0°0 
190°0 
€Z0°0 
S100 
7£0°0 
7£0°0 
L700 
$700 


€r0°0 


(aH) 3-2 


aH 


r81'0 
S610 
760°0 
000°0 
y9e'0 
ZLI‘0 
7L0°0 
7S0°0 
000°0 
6r1°0 
060°0 
rIl‘o 
Spl‘0 
0L0‘0 
7200 
610°0 
L610 
000°0 
000°0 
000°0 
000°0 
9S1°0 
6S0°0 
9L0°0 
1170 
760°0 
170°0 
810°0 
€80°0 
000°0 
7Z0°0 
p81'0 
000°0 
7770 
[110 
9010 
£770 
1610 
rOl‘O 
ISt‘0 
190'0 
000°0 
7800 
000'0 
000°0 
6770 
0010 
v71'0 
$970 
SOr‘0 
910°0 
80°0 
rc0'0 
6661 © L661 
auusA0]] 


bilo I€ es 
€v70 Ic cS 
8£0°0 &c Ls 
000°0 ¢ OS 
L80°0 Ol 67 
000°0 3 8V 
000°0 Ol Lv 
000‘0 € oP 
000°0 G Sv 
aul) €L vy 
ror'o 9L ev 
Ivl‘0 8E (44 
£910 OS Iv 
000°0 0c OV 
LII‘0 vc 6€ 
000°0 LI 8e 


790°0 6 3 
000°0 L 9¢ 
8010 8 ce 
000‘0 Cc ve 
000°0 C ee 


601°0 OS ce 
8110 IL Ie 
L7Z'0 ce Ot 
L810 Sc 67 
000°0 €7 87 
860°0 ce LC 
000°0 9 9C 
000°0 Ol Sc 
000°0 6 vc 
000°0 8 e¢ 
000°0 9 CC 


000°0 € IZ 
€77'0 SL 0c 
I€10 Li 61 
000°0 vl 81 
0Z1°0 LG Li 
E110 6¢ 9I 
000°0 LC Cl 
000°0 (6 vi 
velo £ €l 
Seco II Gl 
000‘0 L ig 
000°0 9 Ol 
€LI0 ¢ 6 
blT70 Ye 8 
L60°0 8L L 
ZI10 Li 9 
810 0c S 
9r1°0 87 V 
€90°0 <6 3 
7 0 (4 
9710 Ol I 
apeg ‘Mu surewog 


0007 29uUUE : sginsse UOU sanbyeISy,p suoyAOdod sap saulewop s}yed 1nod suogewysy 


[ neaiqe | 


600Z uinf ‘ajanbua,p sanbiuyoo |, 


‘soyooid say 
jUuOs GH 1° Gy SuoneuMso so] ‘Z/[ ap UONeIapuod ap one 
un jUuO0 sonbnewuUAs jo sajyooMp suoyeUMsa say puenb arp 
-b-1S9,.9 “¢‘Q = puenb ‘onb Issne suojou snoN “JoUTquIOS 
So] Op JHeoyTusis uaAow unone eae AU [I 39 SayUdIS]Ip 
ZOSSE JUIIL}D STUN | IP NLIAIU Ne S3]QeLIVAOD SIT ‘SaTqIej son 
JUOS UOT[HULYIE Pp Sa]le} Say JOP souleuop say nod saydarp 
suOneUISa say Jodnoisai sed au ap IsIOYd SUOAR SNON ‘saT[NU 
UOU SICUI SaTqIe} JUOS SouUAaISaX4eq SUOTJRUTT}SA Soy onb spur} 
‘Sa][NU JUDANOS JUOS sdginsse UOU souUOsIod ap UoTOdoId Fe] 
OP SaJdaTIP SUOTFRUT}SA soy ‘ayNad say Sd UOT|HURYDE,p aT]Te} 
R] JUOpP souTeWop saj mod ‘anb 3sa gy No GH suoneusa 
sop sodejueae sap ug (4| “dm i<)rea =(qH) 9 
d[NUUOJ ve] Ted saguuOp ywOs 39 (GH) 3-’9 Jed sadusIsap JOS 
GH suoneumss xne sagisosse (uoua}sod e }-"9) uorstogid 
op SoIsoul S3’] 666 666 & 000 00S =P eTI) 9p WSa oun & 
jueusyredde sue / | & GQ op sutddyryd sourwmoy xne puodsau09 
I ‘]]Nu jsd UoTHUeYSE ,p aT]Te} ey] onb sored nyjoxa yso 7 ouTeU 
-OP JJ ‘N00Z euUe | Mod sautewIop syed sIdAIp soy mod 
soxouuos sadAj-sinaua say] 39 Sginsse uoU sanbyeisy .p suon 
-1odoid sap SUOTFEUUTSA SASIOAIP S23] SUUOP | Nea]qQe} J] 

13° ¢‘0 ‘T‘0 = Y¥ smod saquasaid 

JUOS sye][NSdI SY] “Y oNouTeIed np x1oyo sJusJayIp nod qy 
SINdJVUISd Sop SUOAR Snou ‘anno UY 70‘ = P=2 Mod aonb 
soyuasoid yuOs ou yo s1oudiadAy sop x10yo ne sajqisuasut say 
yUOS s}eINSZI SYT “7000 ‘Z0‘0 ‘70 = P - Z00'0 ‘200 ‘7'0 
= 2 :SuOIspisuod snou ‘IoudiedAy anou sueq (GH) 3-'2 
sogjodde yuos saxouuo0s uoUa}sod e sadAj-sinalla say “qH 
sogjodde juos sayja nesyqe} a] sued] “SUOTFRUTSA Sad I1U3}qO 
Inod Z ee PR] & aILIDANP DWOW 2poujeu ey suostyyn sno 
(Ad) are = ee red saguuop juos sayuepuodsaxi0s 

sauuatsakeq sonbryoreiory suoneumsa say “Am f ="d 
Jed saguuop jos sayalIp suleulop sed suoneuNss so] 


966. Vp fu Si Sy 
“in 8g + Uxtg + xlg + = (“d) Bo] = “9 

suOsI|gpour snou ‘gy ascyeue,| og -(“4)7 = "d wos 
‘oureWop Wed ,7 

3] suep oyun ,f ey] op jejo) yermuey nuoas gf = “x 
‘ guIeuIop 

ined ,7 9] suep gum ,f eI ap sapmg,p nesatu af = “x 
‘ suUTeUIOp 

med 1 9] suep gyun ,f eT op ay[rury e op ayre, ep = |x 
‘1 onbeyo 

mod j,=“m'!< onb opos ap “mifzs"m = "mM 
‘ sulewop yyed 47 3] suep 

gum ,f ey e anbydde oseuuojjyueysg,p spiod ay = “Ht 


‘ SIPE[PUI-souRINSse 
aun (apessod) sed apassod ou surewop ned ,7 9] suep 
suuosied .f re] Is Q no [= “A yUOs aseq ap saguUOp sa] 


angdojeyZ0 nv X-100-Z1 ,N ‘epeued onbysnej¢ 


‘Tp OIQUINU a] BL10d G66 666 
® 000 00S eIt@} 9P WS oun suep juRAIA sue HO B 8 ap 
soutddrryd soururay ap anjysuos outeurop yed 9] ‘ayduraxa 
req “("y “*y‘“y‘'y) ya]drapenb ne juepuodsauioo = *y 
+ (1- 9)p +(1— “714+ (I- '¥)pZ ernuusoy RY] Jed gy019UMU 
JUBUSIUIeUL ySo suTeWIOP Hed UP) ‘000 00S Z < NO 666 66h Z 
-000 000 I ‘666 666-000 00S “666 66h] ATI =P (SY) 
sureyjodoneu onbysynejys uorsei oun e yuaryedde suuosied 
re] onb uojas ‘p no ¢‘7‘{="y ‘uyuq ‘snd 34a sue cg 
Sop NO suv pg & QI Sop ‘sue /| & CQ Sap odnolsd ne Juatredde 
suuosiod ej onb uojes ‘¢ no 7 ‘T= ‘y ‘sing ‘auTUay 
aun no swwoy un }sa suUOsied e] aonb uoyas ‘7 no [= “y¥ 
‘sing ‘oJIe[NsUI NO sISY,p ouUaIpuT ‘ouTddrytyd ‘astoutys sa 
suuosiad ej anb uojas p no ¢ ‘7 ‘T= 'y no “(Py “fy ‘“y “y) 
j9]drupenb 9a] suoIOpIsuod ‘s}InQsuod jUOS soUTeUIOp 
syyod say yuouWO0D aluoep sed suoduewmUI0d snoNn 


sonbyvisy soy 1nod 
souIvwmop sj}y0d anod suoneulpsy °s 


‘apaooid nb uoneAsasqgo |] 
osummjuod nb 30 ‘Ego’, e& esa ana alaAe.s ouUMSo Y 
90 ‘suUOIaIgpIsuOd snou aonb sdutexa | sued “a{qeIOAR} SUTOUT 
3] SoguUOp xne o[qeidepe sAneutxoidde oud e uoNNquysIp 
PR] so Y yuedeyduisar Y ooAe ayUR}NSOI UOUd e UONNQUysIp 
eT] 1 op oyooid san juowaTjonyiqey ysa mb (9) ‘7 — (9) 'd) 
((Q)'D=% & gsturxeur so (r1)WO_ ‘pues y mod 
Gree (9)'9 2 (DO =(9)'¢ anb juenbrewor ua si0je ‘7 
dUIDIONY} 3] Suep (q)' ‘O red (+ 1)/-% oP WaloyyYe09 9g] 32 
(q)'¢ red (Y¥+1)/¥ AP WeroyJeo 9] suoUsISep snou Ig 
‘JUBAINS jUTOd 9] 
JOUBI[NOS dp JUSTAUOD [I ‘VI90 JURAR “URpUsdaD “gy SIMo}eUI 
-1189 Sap FIOFW Sep soayeutxoidde suoneumjss sop JoAnoy 
Inod sjejnso1 sad suosonbrydde snou ‘ayuRAIns uoKdes eT VY 
‘axouue Us SogjUasaid JOS saUaIO9Y} S90 ap saAnaid sa] 


(9) ee ‘wi — (gyal aus x 


ft fi fi EN) a 


1 
| fx(g)-5 fx (@)'u— Dig) f+ 
((9)“w(q),-S)4 
3 ),-5—((g)"u- 1) 
((9)'w(9),-%)47 
AGN tab) 


(q) “w((g) "wz — 1) — (gy "ue — Ng)" [rex 


seulewop sjijed unod sanbuidwa sauuaisaeg ja Sanbiyoiessiy SUOHeWUNSY : SUOSIE 39 JOZ}Ey ‘WEY ‘eYyUIS ‘WIy ‘YSOUD ZY 


angojejeo ne X-[00-Z] oN ‘epeued onbnsneis 


aeO 
aipio.| e nbsnf 339909 3sa (,,WWOd op aJURAINS UOT}eU 
-1xoidde | ‘s1ory “(¥)°O=(q)x enb suosoddng °7 auauoay 


(S"p) | xco'm —1)(q) fun Slo. x 


a 


zX 


Roce 1)(q) “a” ae 


uh 


=("“Wwod 


qv 


((q) "ua —1)(q) "ue ue a 


red aguuop 3s2 (,_¥)O ps0] 
e nbsnf 93001109 (WOOF Inod seudoidde uotssaidxo 
aun ‘slopy ‘o> %9>'j>0 no ‘9 aue}suOD oUTe}II9 
gun ‘amauadns suioq mod ja ‘'j ajyuejsuod suleyI9 
aun amouueyur auiogq mod e (q)x op jyowsla onbeyo 
aonb amp-e-3sa,9 “(¥)°CQ = (qg)X anb suosoddng *| awasoay] 


‘soide-1d SgUUOp JOS SdWI9I09Y} XNIP SIXT 
‘OBX9 SIPIO.| ausIsap “°C no ‘(¥)’°Q= 7u onb ayos ap 4 
anbeyo mod 95 ‘u chy anb suosoddns sno “ul y< = ‘u 
wos ‘anno ug *fx “x(“w—])"whI< IZ = x(W-D) 
W X=(M% ¥ ( “hae Me yg oo Va) seg = Al 
IOS “UOTRIOU dp S}UDWIZTS suTe}IO9 suOIsIogId snou onb jne} 
[I ‘Bao JURAR SIR “‘UOTIDAS dJUaSoId eB] SUBP SdUIZIO9Y} xNop 
suosuous snoN ‘nuuos jsa Y onb juesoddns ua al ap 
SO9UIT}S9 PIO So] onb sure “(Oq) souusAour sanbyespenb 
SINILID Sd] SUOANON snou ‘UOT}DeS ajUasoId FP] e ‘oyINSUq 
‘sojuepuod 
-Sdl0d gH Ssuoleujsa xne sonuaj}qo Isule suorjeusa 
Sa] gieduiod suoae snou jd ‘T 19 S60 ‘[{Q=Y 2AeSso 
SUOAR SnoU ‘aoIyIe JUasoid a] sueq] “Y OP xIOYO SIdAIp mod 
‘x1 ap suoneumss sap 19ANON suoAnod sno ‘nuuOd 3s2_Y 
anb asayjoddy,| Ins asodai snssap-19 ayLOap spoyj}ew eT 


vin 
=,,2 1a “nd ap gq moyeunse.| ‘uenbgsuoo seg 
pL os Tae ; 
(p'p) ‘(g)"u— + “4 —= 0 
i ‘ I 
Jed suuop 


iso “Wop qq Inajeumsa,] ‘g sed mayeumsa 399 suousIsop 
snou 1g ‘fue heer etek — uopeumse,p 
suonenbs sap mued e sums yso g ouogq *"x( “wu — 1) “w- 
=qo/"wo suouajgo snou Tea xyibte + 11g ix) dxo 
(q) “wi ‘onbystZo] uoneuaseidal ey 90Ak ‘ano ug ‘(“u—-]) 
“w=("w)4=("4)ea ‘sea 90 «suep ‘onb juowlayeSa 
JousINos ap jyuatauod TT ‘go/“wo-= qo/"'8o onbsind 


= laifaat 


| Ry fi fi 
0= - AY ye ie 
ide [(°4 rea) /( "uu ) K 


u 


L9 


(nuuoo Y mod) ajeumdo 

UONLUINSS, Pp UOOUOJ ke] SIOPISUOD OUOP SUOAR SNON 
‘aagsodoid amp9ooid ev] op UONRdTZIPOU! eT a]ISso09U 
inb 90 ‘sameulq yuos uoneoydde anou suep sagigpIsuod 
aseq ap seguUOp sa] “Y JoUMsa.p sed youtied ou sogsodoid 
uoleuss,p suoyenbs sep osysoidde., ‘juowmwenbesuod 
qnu wos “8 “Z4Q ei Z'Zz_ moo np yuoUIg[9 oulerxnap 

gj anb aos ap g = yo /“*30 = yo /"'8o ‘amemg “4 mog 
“Y ap puedap nb [ ,(d —1 + (s)dxed)]q so 4 ojermomg 
RP] OP IUIZICUW IO] LI Op SJUDWIOUI sap soLQVISUSS UONOUOJ 
ey ‘d mod ojeursreur 10] ouIgUI Be] deAe ‘onbsind ‘7 <u 
daar (d ‘u) ayeruourg Io] oun mod sed asod as ou suletqoid 
XY] ‘ouUsAOU es Jed gsUIQ}OeILD JUDWIDIIUS 3S dIeUIQ 10] 
oun jo ‘w=(d)q=(4)q sioje ‘((m—])y ‘wy)eog ~d 
9 ‘(d ‘{)ulg ~ d| 4 1s ‘anb jsa Ja1yU9A 9] op UdXow o]durIs 
UA “Y ua so{qeyMUep! UoU yuos “4 sap saTeUIZIeUI sameUIq 
B19 SIO] Sa] ayo uq “Y op sed puodap ou (“4)sea onb 
duos ap ‘[—= “a ‘sed 90 sued ‘sareuIq soguUOp sap mod 

anoyos s9UUOTUSUIsSNS spoyjoU ke] “UsWIASNomMo]e/| 
“Opeoj\-JOp[ON, 
ap sumpuos[e.| op osprey e suonenbg sao op suonnyjos 
So] SUOANO] SNON ‘suOWeNbs s3d JUBATOSII Ud (JUd}STXO SaqI9 
IS) Y 12 g ap suOyeUMsa sof suouago snoN * (“3 “'3) 
="3 no 9= "8 !S!a {Zz sed soguuop sioje yuos 
sojeurjdo uoyjeumss,p suonenbs sa] ‘ayeulsreul 10] es Ins 
aseq “4 ap yenusojuowiow .v ap3so (“uw — “4)q = “nt no 


Nt fy fig 
fey fq 


2th; 


onb suosod ‘anno uq 


Lapa sie pa 
| "30 "80 Ls 


qo qo 
Teg 2 be M13 a 


onb suosod ‘sayewmdo uoneumnsa,p suoyenbe 
sg] aringsuos ap ugy (“m) 4(“—-Y/(T+ ¥)- (“ue —"4) 
="3 yo “w—"€="81 sameuowsys sieiq “sues won 
-PUII}Sd,p SUOT}OUOJ Sap suoped snou ‘e[I0 Mog ‘soyeursIeul 
SIO] S90 Op syuoWOU sistusid ayenb sop uoeNyead,| JUSUI 
-onbrun jusratnbar mnb (6g6|[ Uosdu0y] 39 aquIepoy) sTeIq 
sues soyeumjdo uoneunjsa,p suorenbs soutepyiao Ins sagseq 
suoneulysa say joyn;d ouOp suONoTeo snoN ‘aTquinopeal 
Zasse oyoe} oun MUsAdp nod soyeulsieul saoue[quiasielA 
so] mod AW] JaAnoyn 3a sagnbijduioo zasse juos 
SO[CUISIVUL SIO] S90 ‘ayeWLIOU IO] eT Wed ke ‘UdUIaSNaMoTe/] 
“{ sop soyeursieur sto, soy soide.p soumsa one 
JUSAIOp Jo SnUUOOUT yUOS Y 39 g ‘onbneid ua Yuepusday 
(1¢| “ig! "zed 
“rap sakeg ap Iayeumsa | ‘ou0g 


fly Wlro 
guuop jso “nen = 


600Z uinf ‘ajanbus,p senbiuyos | 


(€"r) (Quam 4 ue 
x I 
f] f es )qea fl 
((q) “ul — UVa enon +(q) 77 


red guuop so “1 ap qAT 21 20. 


‘a te ae Cre f 
ieee = ("area ¢ (“A — Y)/(CU)G = 


(‘WO 
(“n)rea = (4A )ao0 § “wu = ("ng = (“A 
suonoyeo snou “1ayUOUL 3] 
mog “1 ap (qn1¢q) sIeiq sues oleoul] MayotIpsid mayyiour 
9] SUIWIOD S9I9pPISUOD 39 JUDUITeS9 ynod uUoIssaidxa aad 
fs + 
gg A se EY = i Ty 

(€861 SLO) Jed guuOp sa 

1 “1 ap sokeg ap moyeumsa,} ploge,p suous}qo snoN 
‘uoIsIoaid ap anowered 9] JUe}9 SUTIOD IaTUWIap 39 


Jaygidiayut suoanod snou ‘Y Ud ayUeSsIOIOSp JUDUID}OLS 1S9 
(“r)rea onbsing ‘(“a ‘9)xeul < ¥ aonb suosoddns snou no 


“(fa = (mG = (ri) seas fw = (nh) 9 


(9) 


(Zp) 


‘(E861 
SLUOJ) SIOTY “Udal] UOTDUOJ eI ISO SF yo ‘ouTeWUIOp yHod 47 
fi 


3] suep oyun .f ke] eB g100Sse uejd op md}09A QI ISO “xX 
‘ ong Covet LP GORONG ‘ f f a 
WE yO Tari us Jal (qix)js="w no ‘9 mod 


(4a Ooo) An — “9"u}y]dxe = (“9)u 


d]Iqeqoid ap ay1suap ey] 9aAe ansnfuos 
Joud 3] SUOJEpIsUOd snoU “‘JaI[NoIYed U_ “sIoy sap sasiadsip 
-Ins OAA-NAA y[twuey oun jyusmNsuos sjl ‘ajquiasuq 
‘Toquouodxs sjapoul np onbruours snowed 9] mod 
ansnfuos 1oud un,p onb tsure sto] sp OAA-NAA [Iuey ke] 
op suoyed snou ‘eyao Mog ‘aulewop ywed ap sauuaAoU! sap 
aq SmMayeurysa sap uonsas ayuasaid vy e suosodoid snon 
‘[ = “9 onb ayyeroues 
ap syed sues suosasoddns snou ‘nesanou aq ‘Q= “A= ‘a 
po NTeA ‘o= >) *,9 “‘onuuod sdURLRA ap s[BULIOU IO] P| 
mog ‘|= '4 32 Q= “A= % ‘uossIog op Io] Be] Nog “[- = “A 
y [= '4‘Q=% ‘TJnowesg op 10] ek] Mog ‘a[MUe} a9 kB 
Issne JusuUaTyedde vues Jo SANeSOU oTeIWIOUTG ‘aTeIWIOUTG 
SIO] So] ‘UOSeJ 920 9q ‘“sUONNJOAUOD sImez onb Isule 
‘qyueses onbijoqiadAy (1A ‘ayjaquouodxa (A ‘anbujsul09s (AI 
‘gnuUOD SOURLIRA op s[eUOU (III ‘UOssIOg (II ‘JNowog (I 
MOARS B ‘9Seq OP SIO] XIS Sat pUdIdWIOD 19-a]]9D “‘OAA-NAA 
gtwuey ey e yueuopedde sioy soy gsuajoeles e& (C86 
‘726 |) SLUOJ] ‘s[nu yUsUIOURYNUMIS sed yuos ou “A ya 'a “PA 
no “friZa + “n+ = (“NG = (“9| “Ase ‘onbuespenb 
souRLeA ap uoNoUoJ op aimonns ej soay (“Q),A 
=("9|"«)q =n onb suojddey (OAA-NaA) sonbn 
-elpenb s0ueLieA ap SuUONOUO} & SaT[aIMeU sol[aUsUOdxa 


(T'p) 


ango[eje9 ne X-[(00-ZI ,N ‘epeued onbysyeis 


Sa][TUIe} Sop assed ve] eB yUeUspedde ayIQeqold ap ayIsuap 
aun no ayfiqeqoid op uonouoy aun yuapessod “4 say aonb 
Issne suosoddns snou steur ‘({°¢) sed guuop isa “4 say mod 
g]]aNUSUOdx<a a|[TWIe} ap sJapour a] aonb suosod snou ‘anno 
ug “(yo Tari 'usT=f) aurewop ined 2 9] suep 
sun ,f ey ap asuodgi ey “4 red suouSisap ‘neaanou oq 


auuatsgAeq anbisidwis uoyneuysy “fp 


“O][Ie] opuvIS ap UOTeLICA 
Op sJUSIOIJooo ap ja sadAj-sinaua.p siyosse jwuOoS sIvIq 
sues smojeuINnsa sao onb sayyjad Is yUOs UOTTHURYOE,p saq]TE} 
Sg] ‘SouTeUIOp S30 ap siquiou Inod ney snjd suuoNueUT 
suepuadag “dm fa" 
red guuop iso “Nn ap yop sieiq sues moajeumsa | 
‘aquoo reg “(€|“n “rang “m “nm “5 £4517 4 (4 | “nl 4 
mesa [e|"t], we  (€| “ng “whiz = [4 | “aly 
JUSUID}VIPSUIUT-JURUDJUTRW ~SUOANOQ snou ‘s[noyeo sad 
SOUT Cer Urry Wen CEC men hen (ets I Ge 
red oguuop jsa (4| “rn ““r)aod «ap opreg-aqu0jy 
ap uoneumsay ‘uyug (An Y< wy) -— hwy, 
yso (4 | “n1)xea ap opeD-s}UO, ap UONeUMSO,] ‘oUIgWI oq 
“od YS ay 32 (€| “gq op oprep-s1u0py ep uoneumse | 
sioje “y So saseIl sp siquiou 9] onb jo ‘ase 4 np med 
eaympoid “1 ap aguuoyMURYDe MeqeA ke] DUSISEp cd 1S 
(9661) uur] 
B INd}O9] BJ NVIANOU ap SUOAOAUAI sNoU ‘aUTYILIOS]e 390 ap 
UOISSNdSIP dUN Mog ‘ssuyseH-soodonspy op sUITUOZTe | 
ap uonesyyn,| wornbar 39 ‘“g mod sed 3s9,] ou ayy stew 
‘plepurys }So g 19 7O ap Sal[auUoNIpUoD slo] sop Myed e 
suo]]Hueyss .p uoyonpoid ey ‘ameurq sed a] Inod sagnwu0j 
‘saquUOUSUISNS Sa]JaUUOTIpuOS sap med e suoT[WURYD9,.p 
uoyonpoid ey] Ins sa9puoj jsa saguUOp sop asAyTeue ANON 


SUOAR | Snou  9UWIUI0DS 


I 1 797 1 1 cn ¢ f 
| aie Jano “Af ~ & 0 ‘q\"¢ 


se Na Nala lee Xanl Xe XGN eet ORO 


¢ (6 ath etre DI ~ € “q‘Q| 79 
PT OF Ad X= Ble aX “se 


Ted saguuop juos 

Saq “72 10 q ““g ap sajyatduioo saT[auUONIpUoO s9] JoANOy 
SUOASP snoU ‘sqqIDH ap IMauUOT]HUeYIE,| JoyNo9X9 Mog 

‘(9 anideyo “966 ) Jouue] jso onbyeid sousr9yo1 

au ‘suoneolqnd sasnaiquiou ap suep sasodxa jsa a[eiouas 

WOOW spoujsur eT “sqqin ep mouuol[nueyoe | suokojdurs 

snou “Iatjnomred uq (WQOW) Aoweyl op soureyo sed opie 

-QUOJ op onbusumnu uoNeISIUI.p spoyJoUI ev] suOsITHN 

snou ‘e[99 ap nal] ny ‘onbryAyeue oiorueul ap a[qeyndexe sed 

ysa,uU Inb san8nfuod uoU suUDISsXheq asATeUR OUN,p WSe,S |] 


saurewop sjijed inod sanbuidwe sauuaisaXeqg ja Sanbiydiesaiy SUOIELUIIS : SUOSIE, 9 YOZ}JEy ‘WEY “BYUIS ‘Wy ‘YSOYD OQ 


anZojeje9 ne X-[00-ZI oN ‘epeued onbnsneis 


(¢'€) 


"OZ 
as n 
[ 5 - ox i-zip-( 79) “ 


ml betial 
aut 


=I 
‘uo ut I 


I 
ft fi n 
as — 9) -|is0,. (0) x 


[=f [=! 


(“ols [TL] » (1 0‘¢‘e)u 
AL 


Jed a9uuop jso ayuT0fuos 
wouasod e toy eI ‘siopy * ,( mee ct" ex suelo ani?'a) 
=9 12 fee WG Sac AA psc ett lasing LC) Sal HOS 

1 <u e792) 
(797 /2—) dxa x ([9)u amlp-e-4s9,9 7/p'Z/2 sanguiered 
SO] SOAR DSIOAUT PUTS IO] OUN jINS 7o 19°, Y “OWSLOSUI 
suuojiun woud e 10] ve] g e sanbiydde jsa no ‘syuepusdoput 
juswlayJarynut yuos 79 ya g onb suosoddns snou ‘uyug 
‘Sued ulg]d ap some aun jso XY onb suosoddns yo ("x 
bree Cy bee Ly cons lly) = LX Mos ‘ano uq ‘(fo ‘ON 
IO] op pit yuos ‘n say anb suosoddns snon ‘somojyegye sjayyjo 
Sa] Juos ‘Nn soy ja “UOISseIBOI op soyourered sap 1m9}d0A 
J] 1S9 g ‘sInajoIpaid op smojooA soz ajUaTeAInbs oJoTUeUT 
ap no ‘aouauiedxe,p urd ap simajooa saq quos “x say no 


(z'¢) bp Pay Sah eae ntg xo 


}So dJOpou np ayueains odejo,T 
“¢ WOT}ONS 
eR] e& soroynomied suoneoydde sap suolalgpisuod snon 
‘gmeuigq seo aj suep “d = “1 ‘oynonsed uq °“9 ap aandofiq 
uonouoy oun ysa “xl ‘(“g| “4)sea = (“Q), sh onbsing -(“9), sh 
=("9|“4)q =" xaumsa suojnoa snou puenb yeisues 
uatsaXeq onbiyoresory aapour ay Inod asAyeue,| Joyndaxa,p 

uosey ke] Op suoyNosIP snou ‘uOT}OIs auasoid eT V 
“UOTeNAIS 3}}99 suep ajqissod sed jsa.u oureulop np 
NKIAIU NB SI[QeULAOD Op JULAIIS 9S Ud sJUSWAIQUIOUEp sap 
UONeSI[OPOU eT “OUIeWIOp Np NedAIU Ne UOU jd [ONPIAIpUT 
nesaAlu ne soafqeueAod sop suopegssod snou ‘a{duraxe 
yuosgid 9] suep ‘onb yey ne yuo aNOWIp eT ‘oureuop 
np neosalu ne soginsse uou souUOsied ap sjusUsIquIOUap 
So] IMS 9gpuocy UOSsIog Op IZepour un 11 JasiTyN 
suoanod snoy ‘(“¥)30, = “9 onb suos ap ‘(“y)uosstog 
~"€ ysa ‘ajonze yuosoid 2] suep jyuoutonbry1o9ds 
giopisuod sed jsa,u inb 4ueyodun sjdurexa anne uq 
“snuUOd JUOS 19-xn90 aonb Jasoddns suoaap snou jo spiod sao 
simpoid mod sgsijyn sjagi sowistuessul saj sed yuassteuu0d 
39U So9UUOP sop sallepuodses sinayeslyN say “uepuadad 
‘gsuod9i-uou Pe] op jo uUOneOIHeNS}sod ev] op aj}dur0o jueUI} 
uo g[dwioxa sed ‘suoneumsa sap onb yuaanos juos ou “mn 
sg] ‘onbyeid uo.nb anreuuosel siojayno} ney [] -y¥ 6 ‘T=! 
yno} mod y= "mn F’ = enb alge ap suostjeuuou 
Sa] snou jo snuuoo yuos “nm oa8epuos ap sprod soy 


6S 


onb suosoddns snoy ‘sjueyodun UONeURA op SJUSTOIJJ909 ap 
yo sodAj-smMoaLa.p stposse JUSWIIT[IMIGQeY JUOS s}DOIIP SIN9} 
-eumsa sag “A "m'=!'% 180 “rd op yoomp mayeumsa,] ‘seo 
30 sueq ‘uoneindod ap suomodoid sap sogigpuod ourewiop 
ap souuskour say ““d“uFi z=“ soumjsa suojyreynos 
SNON ‘olpeyewi-soueimsse oun apassod no sed apessod 
au ouuosiod ey anb uojas (Q no | yNeA jsa ‘ouTeUOp Hed 7 
3] Suep suuosiod .f ke] op osuodgl P| ““€ ‘g1duioxo anou 
sueq ‘(“d)\80, = "9 sioje ‘“d sa0ons ap aiyiqeqoid oun 
oak ameurq ysa “4 1g *“9 sap uou stew ‘“4 sap puedap 
mb aanisod uonouoy oun jso yuonoucy eT ““9"Q= “9 
‘squuojsuey sonourered soy d0Ae JaT[Ieaey jUuoWo ne 
ynod uo] onb sored ‘ay1[e19U98 ap plod sues | e xneB9 sud jo 
snuuos sasoddns juos ‘uorsiodsipins op sanoweied « syIp » 
sot “9 onb sipue) ‘ourewop med ,7 9] suep oyun ,f 
e] op asuodai ey] 4s9 4 Ol ee Pee ee eee 


(re) 89! ule) — 49'G) "9 dxa = (“9 "OS 


Jed guuop js9 o0Ur 
-ered un e& a][amuouOdxoe a]][IWIe} op [eloUes s[9poul uA 


suuaisghveq onbiysje.1g1y osAjeuy *¢ 


‘saxa|duioo a8epuos ap suryjd sap yusWI9}901109 9}du109 
yuouusy Inb sad4}-sinaua sap yinpold ja ‘ayesaut UoNneIopuod 
eB] op jo soddess usd ost ey op ajdwioo juan OAYAAAUNS 
onb IausIjnos op jyuatAuOd |] ‘ajqruodsip sed j1e}90,uU 
Sd[GeUBAOD Sap UOT}DaI9S eB] Inod onbyst3o] uorlssaSo1 Pe] 
‘QyoIOYOoI Op xneAeI SOU 9Mjd9]J9 sUOAR snou no snbod9 | 
V ‘sojqeueaod sou mstoyd mod ogsijqyn suoae., snou 
‘gyqnbua ,p soguuop say Inod suoneumMss sap 3a ssayjodAy.p 
$}S9} Sap yInpold jo oMe9UI] UOISSaIZ9I Op soTapoul sop asnfe 
8 UOISIOA SVS °P OAUATAUNS enpgooid ey onbsing 

"Sd[QeURAOD SUTUIOD 
[210} [eI[TWUey NUDADI J] 39 SOpNjd,p NVdAIU 9] ‘a][TWUey eT Op 
gT[e} VP] ‘OUISLIO.| B BDUUOPIO,p dud} np snjd ud 4IeUD}U09 
jeuly sjapou anou ‘arjayuenbas ampgooid ev] soidy 
“ulajd 
sdui3} e 1ojdwia,p uonenis e] (9 30 [e}0} [ele nudAal 
3] (¢ ‘aJUapsooid souUR | XN}O} SUTeS Say (p ‘SopMyd,p NedATU 
9] (¢ ‘oyfrumey eT op oyytey ey (Z ‘Jessy jeruowLeW 1e9,] (7 
JUIILID SOdI9PISUOD SUOAR SnoU onb safqeLIeAOd xIS sa] 
‘g[qissod s;apoul 
Mo][IOW ne JOALUe yUWITeUTy Inod (¢_Q‘Q ap UOnROTJIUsIS op 
[Ines un d9AR) [aQUENbes JUSWIDIOUS UOTDITIS op snssao01d 
un IAINS SUOAR 39 SaTqeLIRAOD xIS ap gsUaS s[qQuIdsUA 
un SWW0d suOoLIgpIsuod snou onb 90 ap sied soumuos 
SNON ‘o[QNUT 39 JPQIssoduul yso [eNIUI s[apour sy SUP Sono} 
SIMJOUI ST ‘OOS eB ANaLIadns jsa safqeueAOd op siquiou 
3] ‘UONONPOHUI,| suep QUUOTUSU SUOAR,] SNOU dUTWIOD 


SI[GBLIVAO) SIP XIOUT) °7 


6002 uin! ‘ayanbua,p sanbiuyoo) 


“SUOISN]OUOS 
SoUIvUSD ~SUOIN snou ‘/uUONdES RT e ‘UUq ‘sajoamp 
suoneuInss xnv suoreduw0d say snou jo “qq 19 GH sepoyjoul 
S93] JURSITYN Us QQOT US onbNeIse uoNe[ndod-snos vy] mod 
sojuepuodsall0s uOIstdaId op SaIMSoU Sa] 39 SouTeUOP syyad 
Inod suonewnss so] SUOANOY snou ‘9 UOTIaS ET YW WOW $30 
op xnop s1ps0.| e nbsnf 3390109 uoneurtxoidde oun,nb Isule 
‘sosodoid gq smayeumjsa sap (uo, snjd Jastogid eg) xnap 
oIp10,.| e.nbsnf saysauI09 (WA_) souuaXoul sanbyeipenb 
smMoid sop suOANOQ snou jd dsAnRUIAYe gy spomjoUl 
oun Pp SUOINOSIP snou ‘¢ UONdes eT YW ‘asodoid gH 2]epoul 
np uonenbepe | ap suoynosip snou “p uonoas eR] YW ‘sexouuo0s 
UOIsIogid Sp SdIMSdUI So] 19 SouUTeWIOp s}yV0d mod ‘suoneusa 
So] Iuaj}qo mod olessoogu sfeloues GH ssoj;opomjoul 
R] 8p suOINdSIP snou ‘“¢ UONSES eT Y ‘anbyeIse uoNeindod 
eR] Imod sajqetreaod sap uonNs2|9s eR] suopioge snou 
‘Z UONNAS BP] Y ‘JULAINS d] ISA SUODAS seyNe sop uL|d a7 
‘gyn ySO 9]]9 ‘one UN sup onb sipur} ‘| e aye3a 
4S UOT[HUeYSI | Op aITe} e] ‘xno anuS,p sinaisnjd suep Jaya 
uq ‘suonealasqo sonbjanb aonb juauuanu0s ou souTeUOp 
xnoiquiou op ‘synsodsa1 soulewop sma ke sagjoaye 
yuos souuosiod sa] puenty 96=pxPxXZXE eB esa Js 
SOUTEWIOP ap ]e}0} sIquIOU 9] “OUOC “(000 00S Z7< 666 66 Z 
-000 000 I * 666 666-000 00S ‘ 666 667 >) SUE TodonsuI 
onbysyejs uoisel ev] op ayie} ey] uojas suorse1 onenb 
39 saovl anenb ‘saxas xnap ‘(snjd jo sue cg 30 sue PO B QI 
op ‘sue /| & 0 op) o3e,p sodnois ston 94STx9 |] ‘UsUUSTAOId 
SO]]2 NOP UOISdI e] 39 OxOS IMI] ‘ade1 Ma] ‘ade IMa] UOTaS 
Slonoed soureulop sap e sag}oayje jyUOS sauuOsied say 
“919 ‘suoTWRAND sa] ‘suBoUes say ‘suatemeyH] So] ‘steuodes 
So] ‘suotWeMaIA Say] ‘suadI0d say onb sya} ‘sanne.p (Pp 
12 aISy.p susipuy sop(¢ ‘surddiryg sep(Z ‘stouryD 
sap ({ asoduros yusuaT[auULI0J yso onbyeise odnois oT 
‘snjd wou a]917Ie | suep asATeuR 39]99 
sed suonjout,u snou ‘acdedsa.p ayney ‘onbyerse uonefndod 
ey mod onb suiow ey wed dIsO;opoyjou ke] onbsing 
"OEE & Tesd }Ie}9 souTeWOp syned ap siquIOU 9] ‘sed 99 
sueq ‘onbruedsry uoyejndod ey] mod ayqejquias asAyeue oun 
91Nd9xKd SUOAR SNON ‘OOOZ a2euUe,] Inod aonb sajuasaid yUOs 
OU S}e][NSBI Say “UOISIOUOD op Tonos Jed stew ‘QOOT B& L661 
op souue onbeyo mod samoayja aj0 & asdyeuR, JT “UaIsaheq 
enburduia sjapoul ay ms onb tsute ‘uatsoXeq onbryoress1y 
Q2poul I] IMs sagseq JOS UOISIOQId ap SamMsoUl so] 39 
suojeuljss sx] ‘anbyeise uoneindod x] nod so.uemsse oun 
Jed say2anod uou souuosiad ap uotpodoid e] op sutewop 
Jed suoyeuyss sap uononpold ey] Dep spore yUosaid aT] 
“mM sajtyo 1 “°g ap 
smMajea say GuouTUaptAa ‘uojas “Am ap juepuodsa09 
sokeq op onbsu ne mousdns uaiq zasse ang ynod 
mb‘ (% +1) = °¢ “(d—- 4) Ca - 9) +(Ca-1) 4 sokeg 


angojeyed ne X-100-ZI1 .N ‘epeued onbnsneis 


ap onbsu un opessod g op ng+d4(g-—][) maeumsa] 
sige ‘“Y ap JUSUIOLIOY oyeoe S Pp no ‘“(u‘""*‘~T=2) 
‘(V ‘T)N 10] 9p senqinsip yuouonbyuapr ja syuepuadeput 
yuos ‘g sa] Is ‘ajdutoxa sed ‘ayaa sed ysa.u gsoddns 
gjepour a] Is Yuepuedsag ~ [mt -ng+4(g- DF 
+(g-1),_u seseg op anbsu a] apassod [= m'/% soar 
‘9 ap ‘A'm'/< Imoyeumsa,] ‘smarie reg ‘asoddns sjepour 
3] snos (q —[),_¥ seAeg ap onbsu a] apessod imayeumnse 
we) (WK +D=_a no ‘Ng +4(g-D=(Ng+ (g-D] 
lak etonsouig Keeui= Q ep (UoLajsod e suuahoul) sakeg 
op InsyeuUNysa,| sIOTe “(4 **"" “T= 72) “(py “MN TO] op sanquysip 
yuowonbyuspt 39 syuepuodsput yuos ‘g say onb sipury 
‘I“@)N 10] op sanquysip juauepusdspul yuos ‘@ | '¢ 
SQ] IS ‘soydumts sauuatsaAeq SUOTIIPUOD Sop sup ‘aTdUIDx9 Ieg 
‘OJUA so onbyamodAy s[apour a] Is ayoVOyJa,p opiod oun 
Joutenus yuoAnad sjI ‘ajapowl np day99,] anuOd juasa}01d 
2193 99 op sMa\eUIT}Sa say onb sioje Guepusdad ‘uosdw0y |, 
-ZNAIOF Pp IMdeuIysd xnowej a] 3Sd onbisselo ajdwoxs, J 
‘urjd ne yoddel sed sieiq sues suonewmsa sop oimnpoid 
Inod sasi[Nn JWSANOS WUOS S|] “‘UOTdITIS ap say{Iqeqoid sap 
ISIDAUL,| B SauUOTOdold yUsWAT[aMAIGeY JOS SoT[ONprATpUl 
aseuUO][HUeYdS,p soyUN xne sol] asepuos ap spiod sa] 
“osepuos 
op ued 9 sed a9)SISse Jo dJapoOwW UN INS sgpuo} UONEUITSS 
oun dUTWIOD adI9pIsuOD ano ynod spoujoUT aNoU ‘suas 
uleyia9 uN suep ‘UDG ‘souTeUOp s}jed mod suoneumMsa 
Sg] Jajnoyeo nod ajapow! ay aonb sure asepuos ap spriod soy 
asAyeue 31}0U SUeP SUOSTTN snou ‘sInayne soo SWTWOS ‘ano 
uq ‘oeUIg sed a] Inod (pOOZ) IEW 39 YSoYyH sp myao onb 
a,dums snjd nad un sa asodoid ayapour a] snos suoneuUIss 
sop souusfour sanbnempenb smoa1a sop nzed I] “(€R6T 
‘TQ6] SLUIOJA) Sonbynespenb soueueA op suOTDUO} & T[OIMjeU 
gyjonuouodxs ayes ey e s[qeoydde jsa oy[q ‘someulq 
sosuuop xne osoyuny sed jso,u a[eIgued dId0[opoyjoul 
eR] ‘sinajne sod ap aJ90 suTWOD ‘anno UY ‘SIMo}eUIT}so 
Sa] Ja1oqe{a Inod (pOOZ) He 19 YSoUH ap ompesdoid ey] 
B SUOTIOTJIPOUI souTeyI9 Joyodde suoasp snou ‘Juanbgsuos 
eg ‘oyun, | Sp Nedalu ne saguUOp sap ns otndde_s sysoidde 
ayuaseid e] ‘UOIZ91 e] Op NedATU Ne Sa9UUOP Sop Ins s9puoy 
189 (O07) HIE 39 YsoYH ap ay[20 aonb s1oTy “(p007) HRW 
32 YSOYD op 12 (0661) O&Y 39 peselg op sayjoo e onsoreue 
anbnoydusAse aspoujsu oun sed juepuodsauios suushoul 
anbyezpenb moa] op syyeurxoidde smoajeumss so] 
19 Gq SINd}euINSd sa] suOUI}gO SNON ‘soyewNdo uoNeUWNsa _p 
SUOT]OUDJ Sap aLIO9y) ke] Ins sapuoy yso gy oyooidde,T 
(2661) UpeD 32 pnons ‘uefereyeN ‘YsoyH sed s9gsodoid 
39 & spoyjsW ey ‘uonendod ap sourewop sjjod mod 
suon1odoid sap wouaysod e sodAj-sunaua so] 39 souusAOUI 
S3] Iuajqo mod (japow paxiu Avaul pazypsauas mod 
WINTD) 9sifesguad ayxTu ameauT] aJ9pow np qH onsoyeue 


soulewop sjijed nod sanbuidwa seuuaiseAeg 38 Sanbiyoiuessiy SuONeWNSy : SUOSIed }9 HOZ}eY ‘WEY ‘EUUIS ‘WIy ‘YSOYD gG 


‘SONSHRIS YITVOP{ JOJ 19]U9D JeUONeN ‘suosieg “J URA { SONSNRIS YI[LIH{ 1O0J JaUaD [eUONeN ‘Jjozey uoIAP * AVIsIOAlUy) 3eg UesIYIP] “Hew 
eyeiqedey ‘ sojosuy soy ye eIWoyI]eD Jo Ajisioatuy) ‘eyUTS Iqeiey ‘ AjISIOATULA [eUONeN yoodsunhy ‘wry oye ‘ epuoys Jo Ayissaatup) “ysoyH AejeW “| 


un Ins d9puo} jso asdyeuR,’T ‘SoxoUUOD UOTSID9Id op SoINsoUI 
sa] onb isure ‘sautewiop sjjod mod suoneumjsa so] Iu9}q0 
mod (qq) sauustssheq sonbuidwia 19 (gH) souuatssheq 
sonbryoreisty «= sopoyjout «Op )=6SUOAIOS «=SnOouU = SNON 
‘soutewiop 
sonne.p e soguuop op juniduia sed uorsisgid spurs 
snjd oun ,p ourewiop sed suoneutjss sop JUSWAT[aNIIGeYy SUOp 
yUasInpold ja souTeWIOp Soy] BUD UDI] UN JUdSSITGRId SOTOpOU 
Sag ‘sonordxa no saylordwit sajapowl sop ims jueseq 
3S Ud SOUTLUIOP S99 dP Sd}D9IIPUT SUONeUISd sop jUssInpold 
nb ‘sourewop sjjod mod uonewmnss,p sopoyjsul sop 
B ILMOIAI SIO]TS Ne} [[ “‘UONLUA dP SJUDIDIJJI09 spuvss op jo 
sadA}-sinaa sapueIs op soosse 39,p 19 ‘soTqely say ang 
sed ou ap juonbsu sayoorp suoneumjss say ‘ouod ‘apueis 
so sinofno, sed jso,U souleuop soy] suep uUOT]NUeYD9,p 
ale] ey] “OSTA amleyWOuTUT uoTe{ndod-snos sun mog 
“QIAI]DI]JOO B] Op SoIe]MUIs someyWOUTU sodnois 
sap jo sonbyeisy say ‘sanbruedsty say onb sja} ‘sozatpnored 
suonjejndod-snos sap ims osaxe 4sd asdyeue], puenb 
Sed do] JUOWOMessao9u sed jso,U B90 ‘aNUOD Jeg ‘oUTeWOp 
onbeys mod opueis JUSWa[qeuUOSTeI Sd UOT[HULYDI,p IIIe} 
eT onb oozed ‘sajqely zasse yUos ‘sasiapuod uoyNueyos .p 
suorplodoid sa] ai1p-e-}s9,9 ‘souTeuOp $39 mod sa}daNIp suon 
-eUINSd So] ‘oUIeoUIOWIe UOTVe[Ndod ey ap s]quIasUd | Nog 
“sonbrwouo0d9g0190S Sin9}ORJ SdNe XNdIQUIOU dp 39 (3}STXO 
ud ]I,S) sowoedeour say ‘ayues op souatqoid soy ‘nudAdl 
op d089}e9 eT ‘sopnjd,p NBdAIU J] ‘UOISSI ey] ‘ddeI BT ‘axS 
3] ‘ode, onb saya) ‘sonbrydeisowrsp sanbysuajoe1ed so] 
Ins uoneuLoyUI,p suosodsip snou ‘snjd uq ‘uouU no sIpeyeur 
-20uRInsse suN apassod suuOsiod sun. nb je} a] ap-e-1s9,9 
‘grediouud asuodei ap a[qeueA ek] INS s}udWIDUSIOSUDI Sop 
suopessod snou ‘souuosiad sao Mog ‘saTqeueA QOg op snjd 


‘xnop o1pso.| e,nbsnf joexa £ suuaAow osnbyeipenb Ina ‘ steiq np 9309 ¢ onbyeisy 


Ins ja sauuosiod YOO OO 2p snd ms saguuOp sap judT}UOS 
gjeuisiio ajanbua,| ‘sous1gJo1 op s9uue 3)n0} Mog 
‘aynpe un red sarmmoj jUos sasuodal say 
‘sjURJUD So] INOg ‘asTuLod jUSWATeS9 ySso s9sod1ajuI suUOSIOd 
Jed asuodal ey] stew ‘MdIAINUI.| & Jodionred e soyIAul jUOS 
J]IOTWOp e saynpe soy sno} ‘sodURINsse oun Jed ampIDANOd 
PR] Ins sayjao aduroxa sed ‘« yermuey » addy ap suonsonb 
Sa] Inog ‘souuosied QOD OOT UoMAUD auasgidoI mb 99 
‘XNeITIWIeJIUN SaseUdU Sap JUOS % Q6 op snid jUOp ‘soseusUT 
000 Ob UoMAUd.p soidne soaryionsar jos STHN ® op 
saguuop sa] ‘aauue onbeyD ‘(Jpd'arggdy/sqnd79Qz/poid/ao3 
‘SNSUDO MMM SSP] & JUSUIOSUIDOI 9] INS yUOWMIOpP 
np zc onideys ne oyoydxo snd oloruew op soruyop 
JUOS SaT[aIMMONS sgiIUN SIT) “d][TUeJ uN JUTUIOD SQIOpIsUOS 
JUOS UONRISI « sOJ» oUN a}sTxd sjonbsay anue xnad 42 
‘9BCUDU UN SUTUIOD dIDPISUOD }Sd gITUN onbeyo op soiquioul 
Sop o[quiosus,J ‘jUOWOsO] sp soyuN.p uUOTTHUeYOS uN 
SIHN 8] Inod suuonsayas uo ‘onbidAy sguue oun juemNqg 
(0002) suosreg 19 ALLOY “aI00/ “URUOY suep aLID9p 
ISO STHN ®] Op Uondaou09 eT “WhY’styU/syoU/AOS opo" MMM 
//:-dyy assampe | @ Sonus}qo a9 JUdANad saguUOp sa] “NedATU 
399 B Sa]qeLIeAOD soy onb Isure ‘(arpeyeul-souemsse oun 
uou no apassod ap-e-}S9,5) NPIAIPUL,| Op NedaruU ne omeUIG 
asuodar ey yosstumMoy (STHN) AdAING MOIAINUT WyeoH 
jeuoneN ek] sp so9uUOp sa] ‘UoUUSTedde souuosied so] 
sjonbxne UOIsal eT ap jd d0eI BT Op ‘oxas np ‘a3e,] op JUOUI 
-9SI019 Jed sjiINQsuod UOS souTeUOp synod say] ‘saleyWOUTUI 
uoneindod op sodnoigs sioAIp suep oipelew-souemsse 
sues souuosiod op uomodoid ey ap sourewop synod mod 
uoneuMss, | Wed opms,] op aediouud uoneanour ey 


uononpoyuyl “Tt 


> $9[9-SJOJ] 


‘onbyeise 


uoneindod-snos ve op sourewop syyjod sinaisnjd inod saginsse uou souuosied ap uomodoid ve] ap suonewnss.p uaXfour 
Ne I9NSN{[I 1S9 sfesgUgs sISO;OpOUJOU eT ‘satUINOJ yUOS souUaAOW SonbyeIpenb sinda $90 ap sieiq Np sesLUOD sInayeuIys9 
Sop jo sonbuiduis susisgXeq sinojeusa sap souusXow sonbiyespenb sineua sap xnop sipso | e,nbsnf sayoexa suoneunrxoidde 
sap ‘anno uq ‘ssenbijdde juos sonburdwa nb tsure sonbrysiei9iy souusisgheq UOTeUII}sa,p Sapoyjsul soq ‘sanbiydessoursp 
SoNbNYSUg}IVIES SON _P 19 9X9S Np ‘93e_] IP JUDWIISIOIO 9] Jed stUIJap yUOS soUTeWIOp syjod sa] ‘saseyOUTU sodnosd SIDAIpP 
SUPP dIPe[eU-sOUBINSse SUBS souUOSIod op UOIOdold Re] op souleWop syyed Inod uoNeUWINsa | ap oyeN a[OIIe JUasoId a] 


gUINSY 


, SUOSIBY “TUBA Jd JJOZIVY UOAATY ‘IVP BJVAQudey] “eyUIS Iquiey ‘WINDY OYTEG ‘Ysoyy Ale 


SIMIVLIOUTU UOTE[Ndod op sodnois soy suep dIpe]|eUl-ddUvINSSE 
sues souuossod ap uonszo0doid vy ap sanbrardwie 30 sanbryd1es191y 
souusIsgAvq SouIBUOp s}Qed Anod suoHneWs| 


ZS 


anBojeyeo ne X-100-Z1 ,N ‘epeued anbysyeis 
O2-2¢ “dd ‘| .N ‘SE ‘1OA 
600Z uInf ‘ayanbua,p sanbiuyoe) 


‘617 
-LOZ *9 ‘SouSIIDIS DIMO0U0DT pun ssauisng fo }DUANOL, “SAO 39104 
-10QP’T SSOIN SuNeUINSS UI o[duexo Ue UIIM eyep [oued [eoLIOs|}e9 
ut gsuodsaluou as[qeiousiuou BulJapoyy (8861) “Wa ‘Auseis 


‘Li-Tp ‘1 ‘uoupid0ssp jooysyDIS udd14auUp ay fo jousnor 
‘AQAINS 9910,J 10Qe’] uRIpeUeD dy} Woy a]durexs uy :asuodsaruoU 
YIM eyep joued Suisn MOY ssois BuNeUWINsS| (9861) Wa ‘Auseis 


‘$STZ-SIZ ‘7 ‘UONepuNo, aes [JOSsny : YIOX MON 
‘(UIE “FY 19 JOWIN] “4d ‘spq) ‘puawmouayg aaysalqng suidaaing 
sueq ‘UOTeN[eAd puke MOIADI YW ‘sopmIMe UON (P861) “AL “UWS 


LLS-€9S ‘8p ‘sousunyg payddy 
‘eyep UONUDUI SUNOA Jo siskyeue oy) UT dsUOdsaTUOU 29]qQeIOUSI 


“WOU 10J SUMOTTY “(666 1) 'S'd ‘PWLLD 39 “f'D “IOUUTYS “A A\'d “HS 


"878-778 ‘06 ‘UOYDIDOSSp JDIYSIIDIS uDIdauUp ay) fo joUuANnOr 
‘Qyosiqojd ueIUdAO[S dU} JO ased dy} :sasuodsal AdAINS ,MOUY 


1,U0q,, SuljpueH] (C661) “A “eAoys,A 19 “SH ‘Wag “gq ‘ulqny 


'8L6-L96 “6€| “Adojonuapidy 
fo joudnoe updidaup ‘steyyid oy} ploae ‘syyoudq ou} deal oO] 
:yoreasal s1d0jo1wspida ui Burljdures suoydajayL (7661) “AU ‘ounog 


‘SZETIE “Eb “Aj4auupnG uoturdo o19qng 
‘AZojopoyjow AdAINs uONda][9 Ul Warqold uIeYAaD (6/61) ‘d ‘Allod 


‘069 I1-6LS1 ‘pS ‘Satajawoi1g ‘“asuodsoruoUu 
J[QeIOUSIUOU YIM REP [edLIOSI}e9 0} yoeoidde uy (8661) “L “ed 


‘TS-Ph “68 “UO'UDID0SSy 
[DIYSYDIS ubpridaup ay) fo JouUAnor ‘asuodsarUOU 3]qQeIOUSIUOU 


YUM eep [eoU0sI}"9 JO} S[OPPOoW (F661) “AW “UMoIg 19 “] Wed 


"E€t-OTP OE “Ap4alaonG pooyjog usajsay 
ay ‘uoNeIo]dxe uejenzousA YW ‘s3umas 3uN0A Aloyepueu 


Ul sjuapuodsal paploopun su (E861) “AY ‘Jouuo;,.O 39 “fq ‘SIOA 


‘667-777 ‘HI “Yosvasay 
uoiuidg a2qGnd fo jousnor jouoyousajuy ‘Kaains uotuido o1jqnd 
® Ul sjuapuodsal poploopun Jo Joljisse[o se yOmjou [eInoU & JO 
Auyiqisea{ (1007) ‘OD “Bwoyes 19 “yf “eloreD “YW “WIT “D “ejorUO] 


"67-SI ‘OS ‘sousuoig payddp ‘ases aylosiqo}d ueTUsA0[S 
sy ‘saqqey Aoussunuos ajojduroour Joy siskyeue AjIANISUdS 
‘(100Z) “A ‘Maqay3i005 19 “OW ‘premusy “DH ‘sysioquojoyy 


"p6-S9 ‘97 ‘spoylayp pun A10ay] :S2YsijDIG ul SuoyoIIUNUWUOD 
‘yno-doip wopuerzuou WIM eyep [eouosayed = [eUuIpnyIsuo] 
JO UONRUINSS 9ANI9}01g (L661) ‘HD ‘SysIaquajoy 19 “g ‘S[SIYOI| 


‘69E-THE “69 “A4a14ONG 
uoiuldg a4qnd ayy ‘Koeinsoe |jod Jo ainseaw Mou ke 10j esodoid 
pue MatAel VW (S007) ‘OD ‘Apouusy 10 MW ‘Hosnerl “w'q “umeEW 


‘OUT ‘SUOG 7 ABTTAA WYOL : YIOX MON “UONIPT guy? “P/0P 
Suissium ysim sisdjoun poousyoig (7007) “A'C “UIgnY 32 “Wf ‘aTHTT 


‘aBES : JCD led AINQMON “UOWIPA yu9Z ‘uolsiadadns pun 
‘uoyoajas ‘Surjdups :poyjay kaaing auoydaja (€661) ‘f'd ‘SeACIACT] 


angojejed ne X-100-ZI .N ‘epeuea onbysnes 


"$09-685 ‘6S 
‘Ap4ajAonGy uoiuidd IqQnq “SUOId9]2 [eyuaptsaid 766] Oy) SuLNp 
sjjod ,aeay yen, Jo Aovinsoe oy) Jo siskpeue UY (p66) “WU ‘Ney 


“LTI-OLL ‘L “yoavasay uoiudg ayqng 
fo jousnor jouoypusajuy *s{{od uonse[9-a1d UI $19}0A payturOUN 
Jo} [00] uoNdIpoid e se sisk[euR jUeUTLIOSIQ (S661) “L ‘“WITy 


‘OUT ‘SUOS 2 AZTIAA UYOS : YIOX MAN “Sdaaung maiasajuy 
pjoyasnoyy ui asuodsauon (3661) ‘dW ‘Jednogd 12 “Wy ‘seAoIn 


‘DY /MeH pue uewideysy : yOX MON ome auig tg. i pjoqd 
uvisadog “(p007) “AC “UIgny 19 “SH “Wag “fp “UID “Yy ‘ueITEDH 


OL-LS “09 ‘g Hla1z0g poousuvig podoy ays fo jousnor 
‘gsuodsol-uou s[qeiousi-uou 0} y0fqns eyep Aaains jeo”osaye9 


Jo} sousIayUT paseq-[opoW (8661) “TMU “WS 10 “pp FaysI04 


“LOE-E8E “Op ‘Ap4azsonG 
uoluidgE 21q9N_ ‘s|jod uoNd9]9-s1d UI SINOA PpoplospuN BULAzISsEIO 


(7861) “UL “UeUND] 19 “yy ‘Iexpog ‘y “ueWOsIM ‘] ‘YoIMUOT 


‘LET-LET ‘Thr ‘Yoavpasay jayanyy fo jouAnos pouoDUAaIUT 
“SAQAINS UOIII[9 Ul IOUS JO SIdINOS JOYJO pUe SI9}0A paploopun 


Bulonpoy (0007) “Jf “SW ‘Poe To 19 LT ‘Ayfouueyy “py ‘Apjouuey.z 


'8E-1 “6€ “€ SAQS “Kars0g jwousuDg jodoy ay) 
fo [DuANor “WIUOsTe PLA OU} eIA eyep ajo]durooul Woy pooyr[ayt] 
umuixey “(LL61) WO “Wqny 12 WN ‘pley “ay “asduiag 


‘TrI-6ZI ‘SI ‘sousuns jo1iyfC fo jousnor ‘Kaains 
[euoTyeUID}UI Ue JO Spudy asuodsal ;eUOTeUWID}U] (6661) “M J99H 2d 


'8/-89 ‘QQ ‘UOLIDIDOSSP [DI1JSYDIS UDILIAUP ay] 
fo JouANOr “UOISSdIB91 DSIZO] UeIsaheg BuIsn sajdures-asn sI[qng 
snsud,) Ul sapod uolednos0 pue Ansnpul jo uoneyndun sjdninyy 
(1661) “a “Zinyog 19 “N “oyuoyog “Gq ‘Uqny “D°D ‘880ID 


“LIL-IOL ‘pp ‘Jousnor jorid4jawoig 
‘gsuodsal-uou = a]qeJOUSI-UOU §=YIM = UOISSaIBoI = [ROL 9}e9 


ul AWIqeynUep! pue suonnjos Arepunoq uO (7007) “S'd HRIO 


‘AUSIOATUL] 9121S OLYO OL ‘soNsHeIS Jo UoUTeEdsG 
‘anbiuyoaj j4oddpy “Suyseds10} UONdI[9 UI Spoy}oUI eyep BUISSIUI 
BUIS(] :SI9}0A paploapun Sul[pueH (€00Z) “Wd “Auseis 19 “TO “UsYyD 


‘sonsneis jo ydaq ‘oses1y9 
Jo Ajisyaatuyy ‘atjqnd uou 3e10}90p op allows] ‘sopqei AouosuTUOS 
Ul eyep BSulsstu pue saiouonboy dn-poxip] (7161) “L ‘uoyo 


"LSO-€p9 “TI ‘aulaipayy ul SoustIDIg 
‘sajqey AouasuyUo0d AeM-OA\} UI SJUNOD SUISSIUI JO} SOJBUIISA WLIOJ 


-paso[D (7661) “Y ‘ueTuOUTISINGg 19 “AA “JadIoquasoy “D'S ‘Isxeg 


169-C9 
‘69 ‘UOYDIIOSSP JDINSIDIG uDrILMaUp ayy fo jDUANOr “asuodsaTUOU 
g]qeioustuou 0} yoafqns suW0djNO YIM sa[qeUeA [edLOsa}e9 
Joy siskjeue uolssoisoy (8861) WN ‘PileT 1 “O'S TOI 


‘OUy ‘SUOS 2 ADTIAA UYOS : YIOX 
MON UMPA guyt SISAoUp DIO Jpr1408a1DD (7007) “VW “NSeBV 


a1ydeasolqrg 


uonedijdde sane ja|dwoou! 894jUa ajqnop e aduabulj}U09 ap neajqe} un snod sauusaiseAeg SapoUjyeW\ : Hd 38 !OYD “OY ele 


anZo[eje9 ne X-[00-ZI oN ‘epeueg onbysneis 


‘(OETZ8OM) 9910 k] Op ISIOATU/] | 
ap uoNUsAQgns oun Jed ag0uRUTy 919 B& opm oJUESoId eT 


SJUIMIIIAIWIY 


‘IBIE]D BOIS UN SUeP JUIWIDINILI}I[N UTOS daAB Sood 
-dojaaesp 349 JUOLMOd suoysanb sad ‘suoNeLIRA op jo sIeIq 
ap SddINOS sayne.p jo asuodal op sieiq sop Issne stew “UO} 
-elgpuod ap sinajoe} SIOAIP JUSUIZ[NIS UOU aMISUTI eB IOI 
-eul op aipuojoidde ano yWeunod UOIssndsIp BON “UOTDeS 
ay20 suep agjuasaid ajjao aonb asnaimosu snjd uorssnosip 
oun jualainbal sayja ‘uepuadag “77 uONdes eB eB UONeI 
-9puod op suoysanb sap JUSWISA9LIG 9}NOSIP SUOAR SNON, 

‘TEUTUTOU SINJIDANOD ap Nedalu np saysoid 
snjd aimpaanod ap sayyiqeqoid sap 3a synod snd sieiq 
sop ‘saqrey snjd PIO” sop sed sasugoeres yUoWITeIOUNS 
quos sji.nb suas 99 ud AJ Np suoneumsa saz onb 30 
somoiigjue WOLd e SIO] SION saz aonb s}ey[Nse1 somMoar[Iow op 
yuouuop Loud ke sto] sayjaAnou xnap so ‘a}TUI] UOTNTOs oun 
nal] @ puenb ‘uepusdag ‘samougjue sopms xne JUSWIOTeN 
-u0d ‘OYUN, UONNTOS sun daAe.nb Isule dyIUN] UOTNIOS 
sues 90U98UTJUOD ap neg{qe} UN Inod AJ Np suoneUMsea sop 
g]]99 onb apuris snjd PW_ oun Mode ysanod souuatsakeq 


SUOTBUNT}Sa Sa_ onb aQuOU! UONR[NUIS Jed opnje oNOU “USUI 
-amMougue sagsi{yn WoL & SIO] SION sap sed 3] SUBp ‘aqNO 
UY ‘S[99I s}eI[NSaI sap sayooid suoNeUINse sop jUssInpold jo 
a]qeious! uoU asuodal-UoU op sUsTURD9W Np 9}duI09 xnoTUI 
yuouus sjI.nb suas 90 ua sayqeuuostel snjd juos uoLd e 
SIO] SO]]aANOU xnap sad anb a[IAeI SogUUOp Sap asATeue J 
‘syuepuodal-uou S93] 39 s}uepuodal sa] aU 
syuslayjIp asuodai op sjyoid sap ajduioo yuouuen woud e 
SIO] Sa]JaANOU xnap saT ‘yo{duroour sanus o[dums e neo[qe} 
un JosAyeue Inod juoWIoMoLIgUe SdSI[NN 939 JUO sone 
SION Sd] 19 XNeIANOU JUOS xno aNnua,p xnoq “SaUdIO}ZIp 
uoud e stoy burs ms jueAndde,.s souusissXheq uoneunsa,p 
sopoyjgut sop gsodoid suoae snou ‘oyIWIT uUonnjos 
oUN,P ddUd}SIXd,| Op yUoWUepUsdapuT sJepoul 9] JoTjIO9ds 
Inod sakeg ap mayory ay aonb a]]9} onbysyeys oun s1u9}qg0 39 
SOU] SUOTNTOS sap sua|qQoId 9] JawWMO}UOD Mog ‘so;epoul 
sop uoneoyioads ey mmod onbysyejs ouros saudoidde 
ong sed ou jnod .5 ‘Ip juowanny (p66] UMOIg 
32 We £ Z66[ ‘[]O9 10 Jaye) simyes s[9poul un,p sed a] suep 
suIQUI ‘Q < ,) JouUOp juaAnod sayTUMT] suoNNfOs sa_ ‘ayTUIT] 
UONNIOS ke] INS JUSANOS JUSMYIS 9S AJA] NP suONeUISS soy 
‘uoTeNyIS 94390 sue ‘a]qeIOUSI UCU asuodal-uOU ap aduasaid 
Ud S}a[dWIOUT daUD d[QnOp e ddUSSUTUOD op xneo]{qQe} Sap eB 
suuatsaheq asdyeur,| op uoneoydde.| sipmya suoAe snoN 


UOISN|IUDD °S 


SS 


(8S1‘0)SS0'0 
(9S¢‘0)SS0'0 
(ZLS‘0)9S0°0 
(€9L0)9S0°0 
(Z6L‘0)SS0°0 
(p73°0)S50°0 
(0Z6'0)850°0 
(606°0)rS0°0 
(1$6°0)SS0°0 
(7Z6'0)SS0°0 


(L09‘0)8S0°0 
(LEL‘0)8S0°0 
(€78°0)190°0 
(9~8°0)6S0°0 
(p8°0)850°0 
(116°0)090°0 
(806°0)090'0 
(7Z6'0)9S0'0 
(0r6°0)8S0'0 
(LS6°0)090°0 


ag0NON 


(L09°0)6S0°0 
(L9L‘0)090°0 
(S6L‘0)190°0 
(7$8°0)090°0 
(6L8°0)6S0°0 
(~88°0)6S0°0 
($76°0)Z90°0 
(616°0)LS0°0 
(1€6°0)8S0°0 
(S€6°0)8S0°0 


(Zr8°0)0L0°0 
(088°0)690°0 
(176°0)PL0°0 
(L16°0)890°0 
(0¢6'0)690°0 
(676°0)ZL0°0 
(876°0)990°0 
(6£6°0)Z90°0 
(856°0)990°0 
(ZL6°0)1L0°0 


qg0NON 


(8ZZ‘0)S90°0 
(L18°0)S90°0 
(€$8°0)L90'0 
(€L8°0)790'0 
(ZZ6'0)790'0 
(668°0)790'0 
(976°0)S90°0 
(656°0)6S0°0 
($€6°0)190°0 
(L£6°0)Z90'0 


(006°0)€60'0 
(€16°0)680°0 
(€r6O)IOT‘O 
(976'0)80°0 
(156°0)980°0 
(9$6'0)960°0 
(876°0)9L0'0 
(176°0)1L0'0 
(856°0)6L0°0 
(€26°0)€60'0 


qa0 NON 


(L10°0)€S0°0 
(ZOv‘0)7S0'0 
(OLb‘0)SS0'0 
(1LL°0)9S0'0 
(908°0)SS0°0 
(8Z8°0)SS0'0 
(176°0)6S0°0 
($6°0)SS0°0 
(0L6°0)9S0'0 
(656°0)9S0°0 


(O€€°0)9S0'0 
(LS9°0)LS0°0 
(182°0)090'0 
(6~8°0)6S0'0 
(€83°0)6S0°0 
(€16°0)190°0 
($€6°0)190°0 
(176°0)L50°0 
(€96°0)960°0 
(8L6°0)790'0 


qu0NON 


(O11‘0)6S0°0 (SSZ‘0)080°0 (80 ‘8°0) 

($15°0)090°0 = (€¥8°0)9L0°0 (3°0 “9°0) 

(L9S‘0)190°0 (658°0)8L0'0 (90 ‘9°0) 

(98L°0)190°0 + (168‘0)€L0‘0 (80 ‘p‘O) 

($13°0)090°0 + (S16°0)rL0°0 (90 “p‘0) 

(9€8°0)190°0 (016°0)rL0'0 (p°0 ‘p‘0) 

(pr6'0)€90°0 (€€6°0)690'0 (80 ‘Z‘0) 

(Sr6'0)8S0°0 (0+6'0)790'0 (90 ‘7'0) OUI] 
(0L6°0)090°0 (096°0)990‘0 (p‘0 ‘Z‘0) uonnyos 
(6560)090'0 (6r6'0)890'0 _—- (70 ‘Z'0) op sed 
(ZLS‘0)0L0°O §~=—- (0760) 8°ZZ (8°0 ‘8°0) 

(8ZL‘O)ILO'O §=— (F760) 96S I (8°0 ‘9°0) 

(OZ8‘O)LLO°O + (9P6'0)S‘SLI (9'0 ‘9‘0) 

(Z9Z‘O)ILO'O §= (€€6'0)ZIS (8°0 “p‘0) 

(€68°O)ZL0'O + = (SS6'0) 778 (90 ‘p‘0) 

(076°0)6L0°0 (1960) ‘ESI (p°0 “p‘0) 

(8€6°0)0L0'0  (O¢6‘O)r'LS (8°0 ‘Z'0) 

(176°0)S90°0 §=(OP6‘O) ESET (9°0 ‘Z'0) 

(€96°0)ZL0'0 + (6S6O)E'8SI (¢°0 ‘Z'0) ouUl| 
(8L6°0)780°0 —(vL6'0)S°68 (70 ‘Z‘0) uoNn]Os 

agtNON ™TNON (XG aX) 


ty iXg Anod (sasgyjuaied 3.14309) % $6 & 9.1N}19ANOD ap SazTIqeqo.d 3a sadA}-s}.1¥99 sap auUdhOJ 


L neajqeL 


600Z uinl ‘aygnbus,p senbiuyse] 


(902 "J 2 gz *Xg ampe 
-1S9,9) SJUSIZ]JIP WOJ JOS sId99pul sina}d9]9 Sap jo syuepuoda1 
sop asuodal ap sjyoid sa] puenb % ¢6 ap [euTuOU aIMjIaANO09 
op neosat 2] onb s2tarey snjd jyudWIa]qQeIopIsuod jUOS 
ggNON ® a¢NON ‘agNON Haba ge op soqypiqeqoid 
So] ‘Jalfnomred ug jusjuoursne * *g yo **q onb omsow 
® ONUIWIIP sIMjI9ANOd op praca ey] eee uonnyos oun .p 
sed 3] Suep snudjqgo xnad onb sayqeis snjd yuauIayoU jUOS 
AW np uoneumsa | op sodAj-syrese say “(, neajqe) np onied 
sWIsTxnap) sJUasaId 9s dU aI] UONNTOS suNone puend 
“sadA}-sy1e99 Sd] 
JUSUITSS-Snos sauudIssheq suOleUIss sat ‘.<7NON Hed 
B.nb dIfIUBIS P[ID “NedATU 99 B SAMILUIJUI JUDWITeIDUSS JUOS 
SUONPUITSS Sanne sop sayjoo onb stpur) ‘4% ¢6 op jeuTWOU 
QIMPDANOD op neaaiu np snd 9] yWaydodde_s nb sayjao yuOs 
qgcNON ®P SINYIBAN0D ap spiT[Iqeqoid soy ‘souuatspXeq 
SUOT}UISS Soy TW “AJ NP UONeUTSd,| op soyTUMT] sUOTIN] 
-OS Sop swig]qold ne snp yuos sodAj-sped9 spuesis sag ‘anby 
-eid ud 9gsi[qn ang moAnod mod pues doy odAj-ye99 un 
opessod AW Np uoneuMss,| “% $6 op [euTWOU smMpIOANOD 
op nesatu np snd a] ayoordde,s mb aja so AW Np 
UOHCUINSS,| SP SINJIBANOD ap syI]Iqeqold ev] Is SUIOUW ‘OyTUT] 
UOHNIOS oun JUSIAINS puend) ‘xneUTUIOU % CG B DOURTFUOD 
Op SaljeAlayut say mod ampsANnod ap xne} Sa] JOS % CG 


ondo[e}e9 Ne X-[00-Z] .N ‘epeued onbnsneis 


B DINUSANOD ap say[iqeqoid say jo ‘sadAj-syreo9 say Jaynoyeo 
Inod (6) suep s9UUOp soULLIeA ap J[NULIOJ eT gsT]WN suOAR 
snou ‘oy ~* *g mod %¢6 eB ampaAnod ap sayITIqeqoid 
sap 39 sodAéj-syes9 sop suuaXoul ve] oUUOp / nes[qe} oT 
5 ee A bc Mt yl y 
(9.0 < * 7,9 00 90 < * ig amp-e-1so,9) syuarayyIp 
yO} yUOS sjuepuodsl-uou sap 3a sjuepuodsl sap asuodal 
op eid soy] puenb AW aj onb amd ysa 47 NON onb sipury 
‘bos ex id? pOS wix “d puenb Ay 9] onb ainayjIow yso 
sa NON ‘5 HOIGE 21 anuoul af aun “peg = # 
~ Rite NO SUOIPuod soy aol ape aah N93] 
ee yuede U UMOI 30 Weg ‘ ( Cu xg tuix ‘d) ap xtoyo ne 
JUST} STW] UOTNTOS suN nal] e fara i ees UMOI Jo Weg 
op XNdo op JUdIIJIp uONR[NUIIs ap sjeyNnsa1 sou satjanbsay 
Iod suostel sop oun,J ‘“dyUN] UoNNTOs ouN,p seo usa 
sUIgUI ‘AJ 2] ONb syej[NsaI smoay[low op youonbyeuisisAs 
sed yuouuop ou souUdIsaheq sapoyjaw say] onb jo ‘ajTUT] 
UuONOS ke] sp JUoWUepusdapu 4arduiooul sayUa s]qnop 
e neojqe} un suep sure sed jso ud.u jI,nb yuonbipur 9g 30 
¢ xnea[qe} So] suep ,°7NON Inod | k somotipdns simayea 
so] Yuepusday yueyOdun snjd jos steiq uos onb usiq AW 
np uoyeumsay aonb aod snjd WOY oun wear .“7NON 
anb uonje[nuls us sopmjs sop red sQuOW yuo SI] ‘a]qeioUsI 
uou osuodal-uou ap s]9poul un snos joduloour sequa 
ajduns e neajqe} un suep aseo Jed snpuaye synoayjo soy 
qoumsa mod ,7NON 9SI[GN WO (p66]) UMOIg Jo Weg 


(es‘zcoe (E6199 (6ETD9I'T (COS‘e)8ph (ZO'Z)I6T (80 “8°0) 

(er‘e)s67 (LOZIOI § (er DeElt (pL‘e)pl‘y (60°%)S9°I (80 “9°0) 

(er‘e)ps‘t (pizocl (9¢eDe60 (Se'e)zz‘e (LOTEET (90 “9'0) 

(60'E)ES'% (LOrr‘T (ZrIDLOT § (OL‘E)9z‘E (S07) ZHI (80 “p‘O) 

(os‘e)8zz (77671 ((8H1)66'O §6((87H)79'T  (BZTETT (90 ‘r‘0) 

(6Er)Sssl (79ZZIT §=(09'T)96°0 §=(OEb)IS'T § (8I‘Z)E0°! (v0 ‘b‘O) (7 pal 1 C+ he 4 Tg} 
(61‘€)90°T (8I'ZLZ7I (8PrDZOr §=6(CO6‘E)9EZ CET‘Z) PZT (8°0 ‘Z‘0) Inog 
(19‘E)S8l (67 7ST (Op‘)86°0 (Le‘e)e6l (11‘7)60°I (90 ‘Z'0) 

(6z‘S)0S‘I (80°E)00'I (8L‘DE6'O (98p)ZZ7I (Or‘Z)E60 (pO ‘Z'0) 

(69°8)IZI (Ss‘h)680 (97'Z)L8°0 §=(99'S)68°0 §=—(6S°7) 88°0 (Z0 ‘Z'0) 

(6s‘aer'z (estzst (9EDrlt (Lz e)r9‘e (96°1)99°T (8°0 ‘8°0) 

(LL7677 OCS‘ Dsp‘t =(BEDITT §6(6EQ6IE (661751 (8‘0 “9°0) 

(ze‘E)06T (STZIZTT § (8r1)96'0 §=((O6'S)6r% = (9T‘7)ZZ'I (90 ‘9°0) 

(ZI‘E)98‘l (60°97 «(SH TIDSOI §6(z9%E)9E7 = (60°7) 871 (80 ‘v‘0) 

(Zre)Bcil (pz7‘7z7‘1 (Zs‘IO0! §=(S8*E)TTt = (LT7OZI (9‘0 ‘r‘0) {Zp a= 07 Oe + Ce + + te 
(or'r6e'l (pLiDso'l (69'ID)L6O (99'r)6rI §(Or*Z)EOT (¢'0 “‘0) inog 
(Iz7‘E)zs‘t (pelt (8rD00T (Es‘esz'T (OWI (30 ‘Z‘0) 

(IL‘e)erl (8EZ90I (L9'D)S6'O §=((Hs‘e)zZS‘T (8 1°Z)P0'I (90 ‘Z‘0) 

(9s‘s)pzl (97‘E)00'T (68I)L6°O §=(CEI‘S)IZT = (LS°7)86°0 (pO “Z'0) 

(96°8)90'I (6s‘r)E6'O (IS‘%)r6'O (OOL)SO'I (LE‘E)66°0 (70 ‘Z'0) 

aatNON = g@NON — ggt@NON = gg NON gtNON ("“#*Xq '8'XG) 


(sasayjueted 3.130 


JUZINSY snjosqe steiq sap sone. sa[) % OZ ep sayueNbueU sinayea ap a3¥}UIIINOd UN SNOSs 9}TWIT] UOBNOS auNINe JUITAINS 9U 
puenb AJA Np uojewnsa,| & JUIWIIAQEIII SaUUdISeAeq SUOHEWIYSI Sap SUdAOUI SNOSge sI¥Iq Sap ja sauUdAOW JOA SIP SONLY 


9 nea[qeL 


uoneoidde sane jajdwoou! aesjua ajqnop e aouabuljuoCo ap neajqe} un inod sauuaisekeq sepoyjeY—y : Y1ed }8 1OYD ‘!OYD 


vS 


andojeye9 ne X-[00-ZI .N ‘epeued onbnsneis 


(8°0 ‘‘0) 
(8‘0 90) 
(90 ‘9‘0) 
(80 ‘p‘0) 
(90 ‘r‘0) 
(v0 “p‘0) 
(8°0 ‘Z‘0) 
(90 ‘Z'0) 
(r°0 ‘Z'0) 
(70 ‘Z'0) 


fa, Stes Sd St yy 4 Wy 71M} 


nog 


(8°0 ‘8°0) 
(8°0 ‘9°0) 
(90 ‘9'0) 
(80 ‘‘0) 
(90 ‘p‘0) 
(r°0 ‘r‘0) 
(80 ‘Z'0) 
(90 ‘Z'0) 
(v°0 ‘Z0) 


{Z iT eee f of <TC pu Ay 43 Tay Bis (Wiyy 


Inog 


(Z'0 ‘Z'0) 


‘ “ (87‘0)S9‘0 ‘ ‘ “ ‘ “ < ‘ Cay 


(pz‘p)98'% (LL'DZO'I §=(eEO)SL'o § (ZL‘s)Eor = (96'TIDEI‘T 
(y9‘°Z)96'1 (Z6'O)SL‘'O (r1‘0)69'0 $= (Er'E)I6'% (76'0)8L‘0 
(1s‘1)szt (or‘ors‘o (r‘oe90 (O6T)OL‘I (Zr‘0)SS‘0 
(ig‘Dset (sso)19.0 § (sE‘o)L9.0 §=(eEZIsl (rr‘0)zZ9°0 
(po'lss‘o (LI‘oO)sr‘o § (1S‘0)z9'0 = (bZ1)86°0 = (LT‘0) 840 
(9L‘0)LS‘0 (LI‘o)zr'0 §=—(r9'0)S9'0 = (H90)pS‘0 (S060 
(ZSI)ITT §6(Lr‘0)z79'.0 = (re‘O)IL‘O §=((80°7 EET (SEO) E9‘O 
(380)69'0 (OL‘O)Lr‘0 §=(ss‘o)s9'0 §=(r6'0)69'0 = (61 O)IS‘O 
(Ls‘0)rr'0 § (rz‘o)zr0 §=—(89°0)89'0 (rE‘O)LE‘O §=—(9b‘0) rS0 
(1€'0)8z'0 (Or‘o)Ir‘O (rL‘0)69.0 §=(Er‘O)Lz'0 = (e9°O)LS‘0 
(s6‘e)siz (L9D)POT (pz‘o)es‘o (6r‘s)gs‘e (EO I)ZI‘T 
(sezzs‘t (is‘oes‘o (ZI‘O)9L‘O §=((6I‘E)L7°7 = (z8°0)S8°0 
(67 DIO (91'0)99'0 § (er‘oeL‘o § (EL‘I)se‘I (7‘0)990 
(19 D)ZIT =(Zp‘O)pL‘0 §3=— (90) 820 §=(L0°7)9FIT = (9 EO) SLO 
(1L'0)69'0 = (90'0)9S‘0 = (9S‘O) ILO = (060) E80 §=(HI‘0)85‘0 
(zs‘0)s9'0 = (LI‘O)I9‘0 = (€9°0)9L‘0 = (L5‘0)69'0 = (ZE0) S90 
(pe1D)s6'0 (8z‘o)9L‘o §(Eer‘oes‘o (Ss‘1)80°I (97'O)LL‘O 
(p9°0)69'0 (Ol‘O)P9.0 = (LS‘O)LL‘O=—((99'0)EL‘O§=—(€7'0) 90 
(o¢0)9S°0 (6z0)09.0 §=(89°0)LL‘0 §=((0Z‘0)LS*0 §=—(8h‘0)890 
(S0‘0)Zr‘0 (9L‘0)9L°0 (ZZ‘O)Lr'0 ~(99°0)89°0 
qatNON ~— gCNON ~~ gg CNON sg CNON sag CNON 


TZ Tl 
wXg ex 
( 11g 1) 


(sasayqjuaied 


313U9 JUIINGY snjposqe siviq sap sone. sap) % OZ Ip sezuenbueul sinapea ap 3d¥jUdIINOd UN sNoOs sajzIWIT] SUONNIOS sep jUDUUIIAINS 
puenb AJ Np uoQeWNse| & JUDWIAQLIaI sauUaISIAeq SUOQEUINSa Sap sUaAOU SNjOSqe sIeIq sap ja sauUdAOW JIC Y sap sonry 


‘(ned yuarazzIp 
sjuepuodal-uou sop jo syuepuoda sap asuodai ap syijoid 
so] onb omp-e-jsa.0) 99 # yx g no gQ# wx onb 
UONIPUOS & AJ Np uoNeuMsa,| anb ajqrey snd PIO, oun 
® WSTNON ‘alnonred ug ‘agsterq yuoulel989] 389 o]]Jo,nb 
steul ‘WOY,| WWeou09 nb 99 ud AJ Np uoNeuMNsa | e 
ajqereduios sa ,°7NON anes onb anuowr ‘ayy uoTNoOs 
ounone jUSIAINS su NO “9 Nes[qQe} og] “SINoT][Ie Ig ‘snjosqe 
stviq So] Inod reJA 4so eID “AW Np uoyeumsa,| onb WOd 
synods One saCNON TOL MCRiON eS Oli UE ke 
oY xg) jnes fut id 39 =" *g op simajea say sayno} toe 
("7NON AIp-k-182,9) AW Np uoyeumsa,| anb s1qrey snjd 
WOd 94un Wo FIZ7NON ‘7g7NON ‘zg7NON ob aquout 
‘SUNT UONNOS oun jUsTAINs Jonbay suep ‘¢ neaqe} a] 

‘yuonbuewl ssguuop op 
% QZ NO sed gj nod onb steiq ap 19 FCO A.P s[yoid sourgur 
So] JURUDUTRUT Ud %OQE ke soJUeNbueU sIMajeA op. roa 
“waomod 3] Jossed suostey snou Is yUa}UoUNsNe S|] *( “aS 
a Xg) dP I9UUOPIOOS Fk] ep Mone sanbujsurds Shed 
JUOS SIvIq SI 19 WOW Se] onb sored ‘o% QZ op sayuenbueuwl 


€S 


¢ nealqe |, 


smgjeA op oasejyusomod un mod jo xg Sey ae 
Inod sed say onb juajuasgid ou xneo[qry xnop Sy] “AW 
np uoneumsa,, onb siqrey snjd usAow njosqe sietq un 
no suusAowl POA 9unN spassod ajuepuodsauo0s suustsaheq 
uonjeunsa | onb yuonbrjdun | e@ samougyur smoyeA so] ‘oUOg 
‘qyuasoid as OU SITU] UOQNTOS suNONe puenb sonel sowigul 
So] sUUOp g negIqQe} a] onb sipue} ‘sayTWM] suONNIOs sap 
yuouustAns puenb (“Z7NON arip-e-1S9,9) AWW Np voyeur 
-nsa,J @ woddes red (,°7NON ‘* ‘zg7NON 2Ip-8-189,9) 
souusissXeq suoyeuss buto sap suaXoul snjosqe sieiq 

sap jo souuakoul PIO Sep soyel so] suUOp ¢ Nes{qQR} JT] 
“sasuodai-uou sap 
uoneyndun p sduasgid ud uoVeUIsa onbeys op sdueULIOjIod 
ey Moa mod {7 <q = fw + ew + wi} mod suakow 
snjosqe sieiq say 3a souuakowl PIO Sey suojnoyeo snou 
‘QUIDUI 9] “‘UONRUIT}S9 | Op afeqo[s soueWLIO}IOd ke] JOA Mod 
uojewnss onbeys op imped e suousjgo snou onb snjosqe 
steiq oyenb soy ms 39 WOW aenb soy ms suusXoul PY] 
suouaid snou ‘sing {7 ‘[ = f 41°" mod qg0NON 
“ aCNON *”“TNON ®P snyosge sterq se] 39 (OA) 

souusfour sonbneipenb simauia say suojnoyeo snoN 
‘Z UNIS PI B SALIONp 
AW tp uoneumsa, op 3a souuatssXeq suoneumss buro 


600z unl ‘ajyanbue,p senbiuyos |; 


sop uefow ne iw aseo red snpuaye synoayjo so] suoumsa 
snou ‘sagisuas saguuop sa] soide.p nuayqo {7 ‘| = ‘f ‘1 
cettg¢ eteltg ecltte MY sues ug -(suotsIogid ap snjd 
nod 666] ‘[[09 19 YUL onb Isure ‘797 S¥e[_ Issne 310A) 
(L661) SYsIoquofoW 19 S[aTYOIJAY Suep guUUOP a1Q]LI9 9] Jed 
SOUTULINJAp 4S9 STW] UONNTOS 9uN,p soUdLNI90,J “o}TUIT] 
uOyNyOS sues sayne OOO | 38 SoWUN, suolNTOS sap sae 
sallojyegye SUOT[HULYDS YOO | SUOTUD}GO snou onb 90 eB .nbsnf{ 
soguuOMUauIsns suoTIpuos say suep {7 ‘| = 7 “yf 2A} 
suo][Queyoe,p siquiou pueis un suosinpoid snon 
‘yUWIdATIEdSal ‘sased sa.qne 
SION SOP d|[Ie} e] SIO} XNap 39 stoy bud e xneso JUSWOATeUI 
-1xoidde yuos [ = ‘y 10 [ = 'y op aseo eT mod saqquenb 
-UPW SINZeA op ds8eJUBOINOd 9] 39 aTIIe} VP] ONb aIJTUsIS e[I_ 


‘oor = Mu = N 
jo 


a F 
eretel | stl 


OTE TT 
a Ma 


T= 


onb juesod ua 3a XY 10 Xq yuesnfe 

usd % OE & 19 Y%OZ & Sayuenbuew sinajeA ap ase}UsomMod 
3] SUOXT] SNON “70 op JUsUIOIOUI Jed 9‘Q & 7‘9 ap SonoWeIed 
xnap $30 JOLeA suosie} sno ‘spuvis yuos =" “qd jo 
“*g sogouiered say onb snjd jueyne,p juorayip asuodai ap 
sjljoid sac) ‘slogpul smajo9]9 sap ja syuepuodal sap osuodal 
ap sjyoud say aqua soudIayjIp ounone e A.u [I,nb oigrusts 


ZZ ly 
nang “Sela eee UG 
OTP leery 4 
seta cytrg fa Vevey, eh tne 
TeTggy Ty L= 11UV yg Ty L=. ,,0? 


ap suonenbs say soide.q ‘uonejnuns red opn9 | mod 

SOUIUIA}9p 99 JUSATOp Sayourered pny ‘xnesatu xnop pusid 
-uioo Y 10 'y “vy “vy sarqetea sap ounoeyo onbsing 

‘uonenuns red apne 9999 suep ¢ ‘°° ‘T = 1 mod 

7q¢NON ® ™ZNON Youranbrun suoseduioo snou ‘ouoq 


Gy Cyl Gye Kl 
SEL nad i eit lage eke 


z I z I i 
ifr Wf + Ad + Mg + OY = (™Mu)aoy 


: (Z s[Qpour do] SIIp-e-}S9 9) JUBAINS 
g]qeious! uou ssuodal-uou op s]apowl a] snos ssuods1 op 
sjoid sIaAtp jo soyuenbueur ssguuop ap saseyus0mMod sIOAIp 
mod 7 uonoas ve] e& Ssayoap souuaisaheq suoneusa buts 
Sap soureulojiod ev] Joreduros mod sameyusws|ddns sosrew 
d9AR ZX Z BOUASUTJUOD ap Nes]qQe} UN SUOISPISUOD SNON 


uonEnus red apnyq  “p 


anso[eje9 ne X-100-ZI .N ‘epeued onbysyes 


‘OPNAIPIO SdAk 9SINOO ey SUTUIOP 
mb jeprpues oj Juatndde sji.s 10}0A ap Muajsqe.s B suT[OUS 
JUOS SI 19 Te1O}DOTO BSINOD UN SUP INdLIgJUI yUe}a SUTUIOD 
Q19PIsUOod 4So Inb jeprpues un yuotndde sjI,s asepuos uN suep 
« SIO9PUI »» JaIMIWIap B sdURPUd} JUO sIM9}99]9 se] ayjanbey] 
uojas omejndod soueAold ey yUoWaTes9 oULTJUOD ead 
‘jeI9UNS IMaMdoid sUWOD AvIPIOD sInod jusuIa[qe{quiasIeIA 
JUOII}OA d19}91d yeptpuLd Ing] yso Inb sed yuenbiput,u sio9put 
$1n9}d9[9 sop yednyid e] onb onbrydun mb 90 ‘Araur03}u0py 
op press] & SIO9pUT sInaydaj[9 sap Injao onb pureis snd 


AAO ANT 
¢ 
V6 OL = igecioxp? = egg (TT 


onp-24999 

SIO} $6‘Q] UOMAUS jS9 Saploap SINI}99]9 Sa] ZaYO 
AJOWIOSUOW Inod usaynos sp xne} oJ “UdWIITGe[qUIOSTeIA 
qUOI3}0A Inb souuosiod say twued ‘smayie eg “UNnTUOS 
ne sinajda]9 sap uonedionsred ey aloe ,p oNA Ud sIdayeNs 
oun dJANa ud anjoul yop AJawosyuoWy aonb amp ynaa mb 
90 ‘saploap sina}de]9 saq Inod ayI1]1qQIssod ey] aonb spueis snd 


ahd 2) aan 7 
(4 
1-L9 © = gezeq-xy? = Tyg gy 


oup-2-899 

SIO} £9‘ UOMAUA,p 4Sd Sed JUD}0A 
au sji.nb ayjao e woddel sed yUd}0A sTd9pUT sMa}de9 sa_ onb 
ayIqissod ej ‘A1awWiosyuO/ ap MdAvj ud smMa\o9]9 soy TuUed 
‘quod ‘aiajaid yeprpues may so Janb sed yuanbrput,u mnb stews 
JUSUA|Ge|QUIASTRIA JUOID}OA INnb slogput smojoo[o soy Tuued 
OPI SWIQUI J] 39 JUOWATQe[QUIASTeIA JU0IN}0A Inb soploep 
$M9}09]9 Sa] Ted AeIpPIOD op Mdavj Us sM9jd9[9,p aIquIoU 
ne AINWUOSUOW SP MdAvJ Ud SMog}de[9,p sIQqWIOU Np one 
3] aqua soUAaIayIp-3o] ev] anuour mb sajoo sop yoddel 
np ounpuesoy gj so =” *q ‘anuoo seg ‘som xne orpual 
as op ayyiqeqoid moa sed yuoumidxs.u mb sreur ‘Arswos 
-JUOJ JUaIaTa1d Inb stogpul simajde]9 soy Tuued One SUIQUI 
g] 9 Aroul03yuopy Inod saprosp smajoeyo so] Tuued « sed yuo 
-2[QeQUIASIRIA JUO0ID}OA 9U >» IND xndd eB « JUSWA[Qe[qUIOSTBIA 
yU019}0A Inb »» xnao ap aiquIOU Np one oJ aque onbrunpjuesoy 
JOUaIA]JIp e] aQuOUT INb sajoo sap Woddes np sumtpuesoy 9] 3S9 


CIT yy CII Nyy p ZZ 


yl 
* 7,9) 1865.0 = Tey Th, 991 7 = at 
19 
. Ilyy yt 
PSTE OM = Trey TT, B01 T = oe 


anbsing ‘2]qeJousI UoU asuoddal-uOU ap Z 9[9poul 
Cy ‘ ileal ; 
gj suep ©" Xd yo “Ag ‘sjuepodun snjd sa] sou) sop 
apie | & daNsoul ang jnod soUdIAIJIp 9199 op InopueIs VT 


uoneoidde dane ajdwoou! 994jU9 ajqnop e aouabulUCD ap neaiqe} uN Jnod sauuaiseAeg Sapoujey\ : 41ed 398 1OYD “!OYD ZC 


onZoyeieo ne K-[00-ZI .N ‘epeueg onbysnes 


(J0SZ). . CL¥0'O)!:. -(ZS0°O) ss CALT'0) = | (0S Zn, OS 2). SOO): (ST IF0) 


: ; ; , 09890 — gg@NON 
OSPS‘I- 7ZLPO'O 18650 = SZEO- = THOTT = HZOEE ~=—OOGFIO-—s POLED 
‘ ‘ ‘ ‘ « ‘ Ci ‘ ) 
(10S 0 (sro 0) (ZIs 8) (071 9 (1L8 8) (966 2 (SI 8) (071 | ater Peepers ioe ie 
88Ss‘I- 7SPO0O- y6SL'7- €8ZO'C 98ISbh I96PSh PEITE  LBP6'I- 
oem | tyly Tyo eal! z ly ty Ty 0 
“tl 1 “i "tt “9 d i9 id d 


(sasayjuaied 3.14u9 sgnbrpul yuos sadAj-sj.1899 Saj) [e.19u3 Ineind01d 
np uoydse9,[ 1nod ajqv.ous! uoU asuods1-uOU ap Z a[Qpoul a] SNos dUUdISIAeq UOHEWISI,p adA} BUIQISION) BI 39 AIA 


Beye bas 
1S9 SIOQPUT SIN9}9e]9 Sap jo sJUepuodal sep osuodal op [oid 
3] aonb onbrjdunt ead ‘stogput simajooy9 soy tuued jUsWIITQeIqQ 
-UIDSIBIA JU0II}0A Inb sayja0 anb sasnaiquiou snd stoj Op‘p 
yuos sed yUSWIZTqQe[QUIASTeIA JUOIDN}0A dU INb souUOsIed say 


Zt ay 
rotate ee ICS Tl, 21 

op NIA ua_nb sipur} ‘syuepuodai soy 
tuned Avip10d Imod 3ua}0A Inb sayjao aonb sasnaiquiou snyd 
SIO} 9p‘Z JUOS AToWIOS}UOP Inod jua}0A nb sauuosiod so] 


117! 


ee 
739680 = (% Ag+ KDE = TW, 21 


op NYOA ua! (7 = ¥) SIOapUT sIMa}d9T9 Say TuLTed 
Z =! sed UOWO[qe[quIasIvIA JU0I9}0A ou Inb souuosiod 
Sa] aonb sasnoiquiou snjd (,/,...9 alp-e-3S9,9) sIoy Op 
yUOS ([ = 2) JUOWO[QeR[QUIOSTIeIA U0I9}0A Inb souuosiod soy 


Ie!@ 
‘g16ET = (XI — Dz = ar BI 

op NIDA 
ua.nb sipue) ‘([ = ¥) syuepuods: say tuued (7 =7 op 
-B-]S9.9) sed JUDSWIITQe|qUIASIIA JUOIO}OA ou INb sayjao onb 
sasnaiquiou snd (9 adIp-8-}Sa,9) SIO} 60°] JOS ([ =! 
OIIP-B-}S9,9) JUSUII[Qe|QUIASIeIA 1U0I9}0A Inb sauuosiad soy 


Ml 3 
‘600 = (ae he Xz = Sane 
oP NSA ud ‘OUD “¥ 39 71 Op d9dXxIj INaTeA onbeyo mod 


CAC! 


a 
ror Tas (oy Ads Xz = Oly, 21 


jo 


T4CE 


the 
73680 = (** id + Xz = Ty 901 


LS 


p nesqe Ll 


yo ‘7 10 £ ap daxy Inayea onbeyo mod 


I7fZ 


P L,I ul 
Soe t= c a oon Ting 2° 


10 


Nl 


ul 
600 = (2% as acon O] 


‘p negjqei np ;7NON oP suoneUss 
So] jues{QN ua.nb JayuoUr op s[loyjip sed jso.u [I 
XIX hg snog ‘axe]duioo surour uonejeidiajut | expuer nod 
xO vig QUID} J] SUOASI[Z9u snou ‘pueIs WUOWISANLOI 19 odA} 
Area uos onb ja ayned snyjd ey Iso ZL 70‘ = “x Ag anbsing 
‘gqqeas san 1so 4 {7NON ‘SInaqte Jeg carpus As yey]ey 
[I sumwI0S purrs don odAj-y1e09 UN yUO AJ Np suoneUITsa 
SO] SoyNo} ‘o}IUN]T UOINJOS uN 9d\sSIXO ]I DUTWIOD ‘[eIOUNd 
maimooid np uoyde]9,] nod syuepuodsax0s sadAj-spede soy 
anb isure ‘ -7NON suoneUse sz] 39 AJ Np suoreumyse 
SI] p Neayqe} ne suojuasaid snou ‘oqeious! uou asuodal 
-UOU Op Z BEapoul I] snos suOoHeUINsSS say] Ins syWN] UO} 
-NJOS eB] ap 9oUuaUNI90,] ap Ja}J9,] JOUTUIeXd,p jo sanouTeIed 
sop suoleujsa xne yey e nb 39 Ud SID9pUT SINa}99]9 So] 
}0 syuepuodal say 9QUS JOUdIIJJIP 91199 JOSI[ENSIA Op UY 
‘(JUdWIATQeTqUIASTeIA JU019}0A Inb sjuepuodai soy 
ZIYI % OO BUD % 67 ap 38a AvIpPIOD e uoNIsoddo sed 
AroWosjuO|Y & JURTTe adeyUSOINOd 9a] onb yuanbiput saguuop 
sg] ‘Z neayqe} a] Suep onb stpur} ‘AvIpso_ & % 9°98 39 AlouI03 
“UOJ & JUOWIZTQe|QUIdSTeIA JUOIDJOA Inb sloapul smoyseT9 
Sop % p61 WoIOaye ~/7TNON  yg7NON ‘tp juowenne) 
sjuepuodal sap a]]9o ap JUSWISYO} Zasse d1a]JIp Jepipues un 
mod siatwiep sao ap souargyoid ey aonb saJgyuT suoAnod snou 
‘SIO9pUI SINa}da[9 Sap 3a syuRpuodsl sap sjUdIgyIp ssuoda1 
ap spyoid sa] Jajayfo1 e saouNsap UoUd e sIo] soy yasITQN 
qg@CNON P z.7NON anbsing ‘suoneumse saqne sop usIq 
ZOSSB ISSNY JUDIIJIP STeUI ‘JOVXS [991 JeI[NSOI 9] JUOWTNIS 
uou juasipeid J77NON P 4 6 NON ‘JeJouas mnamooid np 
UON}99]9,| 9p sed I] SUR] “d}IWI] UONNTOs oun Ins sed juanyIs 
9s ou smMaeA sa] onb sored “Iauosay a] nod uonoripoid 
OMI]IOW ek] BUUOp a2 ‘anuOS Jed ‘ ayTUN] UOTNOS ouN Ins 
JUSNIS 9s sINgTeA So] aonb sored ‘[e19ua3 Inomoold 9] Jo oTeUI 
J] ‘MausAno3 9g] Inod uonorpsid and ey suuop ™Z7NON 
‘suOloo[a Sop s[so1 sjey[Nso1 xne jUuoWsAneIeduloD 


= 


600z un! ‘ajanbua,p sanbiuyoo | 


ondoyejed ne X-100-Z1 .N ‘epee onbnysnris 


eee 


Osh (96°1)0'SS 6°87 (SL°DI‘IL 667 (PLZO9E TILDE OLS (96'1)8'6r (96'D)S‘Ip ™ENON 
Osh (96'1)0'SS 8°87 (8L°I)ZIL 667 (PL'Z)I9E COLTDEEE OL'8 (96186 (96'D)S‘IP Mol 
Sp (96 1)8°bS SE (83°1)8°p9 See (pl‘ze'se (e9'Z)LIOE = SST (SOI)I‘Sh (IOIIS6E = g§zNON 
Sty = (SEDZLS 1°6€ (SL°1)6°09 Lov (79°7Z)90E (LS‘ZL'8T — ss HIZ «(€6'1)9'7H (BBI)O'9E  —-gSzNON 
rtp — (S61)9‘LS O1GE WrlCL OMS er Thr {C9'C)S OE USSC)S Ete WO NEE DS IY LS Ty's Coram ggenON 
Livy (96DESS 982 CLL'DHIL = SOE: —(OL‘Ds‘9E (EN'TDL'ZE — 068 (SOD) 6r (EODE'TH — 43zNON 
Orr (9609S =CLZ = (PLTUS'TL «= TZE_ (SL‘Z)ITLE (B9DL'OE §=—s EZ (9E'T) POF (E6OT)ETH gd 2NON 
86  (€6°1)7'09 O°€Z (PO DOLL 6b (6r‘Z)ZOT (SHZ)B PZ =—- BTLZ_“(1ODP6E (E8'DL'ZE = ™ zon 
« sed JUSUI9[Qe[QUIDSIBIA B19}OA OU >» + « JUSUID|Qe|QUIdSIeIA 219}0A » Sasuodal sop uOeSITNQ, 

uou tno mo Ino SUNT 
Pp LS LE €9 v7 LE 6€ ¢ 0S SP [991 eI NSIY 
E8p  (ZEE)LIS Bie wee o, oO 8sz (SLiple'ce (91'S)6OP 078 (8ZE)ZIS (€0'S)9'0h = ™ enon 
T8b = (ZEEE) BIS 167 = (6S“€)6°OL 8st (OL*p)p‘ce (9IS)80r =: O78. (8Z*E)TIS (€0'E)9‘OF TOY 
Csr . SOS ES Ore  (HS‘€)0°99 L'8Z (LL‘b)9€e (66P)L'LE = L'EL «(COZ E)H‘Le (66'Z)E'BE —_§ZNON 
9p  (OPe)6'ES OLE  (¥9*E)O“ED Ore (7S‘p)p'6e ClO'p)E'SE 98T CIT‘)% sr (L8‘ZE9E  ——gfzNON 
Lisp (Crp'eye'rs OLE = (L9°€)O*ED I'se (1S‘p)eoz (L8'p)o‘se = L‘6T «(80° S‘*bh (S8°7)B‘SE. —_g EZNON 
6Ly  (vE*e)I‘7S 067 (6S*E)O°IL S97 (espace (HO'S)66E OFS (OZ E)L‘OS (10'E)6'OF  —,ZZNON 
Civ oT OL cS GLC UAT OL CL L'L@ (8L‘p)e ve (PIS)I8E- = @ OL: «(LZ E)S‘8h (H0'S)9'0P = dZNON 
er (8H°E)O'LS bre = CIL‘€)9°SL ep (ez pest (Sops1E = 87 (COOTE) IH (SL°*ZEE = ™ZNON 

JUSWO [NYS « JUSUTS[Qe|[QUISSIBIA BIDJOA >) sosuoda sop uorjzesiy oy) 
oujou0g  s13}0q Aeipioy Asawosuoyy  Adsq 19380, «=uBWa[OD any Yel JIYysly 

4dt10sa1 [eigues 1naindso01g dB Al ANIUAIANO) 


(sad4}-s}.1899 Sa] }UOS SasgyUsted 3.13U9 S2IJJIYD Sa] 39 93e}UI9.1NOd Ua sgUILIdxa jUOS 
$}E}[NSI1 SI) YG (LIAL, P 329 YG 9.1G0}90,P OLYO, | PP 3&}4,| PP XNL10}II[9 SAdEPUOS Saj ANS agSeq SUOTIIII Sap $}¥}[NSI1 Sap UOHIIPIg 


‘T Jgpour np ™[O]7 suoneumse 
saq onb sourgur say anbsaid yuos sayjo,nb sozed ‘sazatpnorred 
seo op sopma soo suep sooudoidde sed ayno-jnod juos 
au "ENON 1° ga0NON ‘gg@NON SuoneUIse so] ‘press 
329 Y ‘sajoexd no souLay sed yUOS dU asepuUOS Np jUSUIOUI 
ne soouarajod smnaq_ onb no omepndodun yeprpues un 
Inod indde may] Jonbrput e yuazIsay suas say anb sored o1yQ,] 
op 3€19,| OP [es1o}09[9 aBepuos a] Ins sapuoy opmja.] suEp 
a|qeuuostes ang yned ssuodsi-uou 9un,p 9}I]IqeiousI-uou 
ep esampoddy,| (€00Z) Auseig 39 usYyD onbiput.{ sumuo0s 
quepusdsg -ajqeious! uou assuodsl-uou op s[apow un 
no g]qe1oUs! ssuodal-uOU ap s]epour UN JasI[NN,p JUsIAUOS 
[IS Joutuiajap mod jelsues aiqjU0 unoNe 9)STxd,U J 

‘JUSWIWAIZIJIP Saypio9ds JUsIOs somresUT] 
-Z0] sajapoul sma aonb usaiq ‘”{oO7 ke anbynuapr anbsaid 
30 ™ENON onb sodusyNos ap juRssaigjUI JUOW]e39 
4Sd ]] ‘UOneUMSa sINTWIOp 300 e yoddel sed s3ejuRAL.p 
nad yuanbgsuoo sed yuayjo yo ™ [D7 ap sayooid say jos 
ouusisgheq suONeUITsSa xnap seo ‘ajqeious! asuodsi-uoU 
9p aapoul un snos AJ Np syooiddel os yo onutunp w 
9p uoHeUsa,] anb a][3} uosey ap stuyap yuos 7°7NON 
2 Jg¢NON Inod Me youde sto, say anbsind ‘arpuane 
A; Wley]ey I auTUIOD ‘adnois suterxnap 2] suep ‘ENON 10 
™ IDI ‘ag7NON ‘zgCNON ‘Suoneumse sane ayenb so] 
32 odnoxs sorumaid 9] suep 7 °7NON 1° af7NON ‘7g0NON 


€ neaiqe | 


:sadnois xnap ua ““7NON Jnes ‘suoneumsa ydas so] 
Jasseja suoAnod snou ‘¢ neajqe} 3] Ins jUeseq snou uq 
“9}0A JUS 
-SAOIJJO JUaIeAR « Sed JUSWIITQRIQUIASILIA JUSTLINOA OU » 
sayja,nb giepoap yueAe souuosied sap yednyd ey] is asueyo 
nd jleme aieul Np uONde]9,| sp JUeUsesS 9] onb stpuR} “19}0A 
ap sare[oap syy]Iqeqoid e] yWos onb ayjanb saurgur sa] yUa}soI 
IoLIOSSY) Np 39 Je1aUEs InamMdoid np “MawWiaAnos np suoNnde]9 
Sop syueused soy onb amnpouos ap youIed snou xneva]qe} xnop 
sop uosiereduioo ey ‘sawm xne JUSUWIOATOATo JUSTepual 
as «sed jUoWa[qe[qUiasIeIA JUdIeINJOA» su saq[a,nb 
giejoap yueAe sauuosiad soa] Is ytelassed as nb 99 Moa mod 
« Sed JUIWII|QLIQUIDSTLIA BI9JOA SU » 19 « JUSWII|QUI|QUIOSTRIA 
BID}OA »» Sasuods soy JWURSITWN Ud SUOTIDI[9 Sop sjE}[NSoI 
sap suonorpeid say suUOp nee]qr} np sMoLgyuI oted eT] 
"SO}TUMT] SUOTINIOS 
S23] SUBP LOU NO JUaNIS 9s AJ NP suoNeUWMse soy Is onbrput 
9 SUONDIT9 Sap S]9d1 S}e}[NSaI Soy Issne JUSTJUOS nes[qu} 9D 
‘QIU NP UOTI]9,] B BSINOD kB] op sed 9] sUep 9}d90x—a ‘9}9) 
Ud S}JEpIpUed sWaTxnap 39 JomUad say] aus syueodu sye99 
Sap JUIT2A9I Jo SaQNe sap sun saz soysoid jUos sodAj-sye99 
SY] ‘sasemjusred anua sadAj-spreog say onb Isure ‘sasmoo 
ayenb say mod « yuoWaqe{quiasIeIA e19}0A >» sasuodal 
Sa] JuauIoNbrun juesTTyN ua snuajqo suoNds]2 sop sypeid 
s}e][NSoI So] suUOp ¢ Ne2]qQe} Np smMouedns sted eT 


uoneoijdde sane jajdwoou! 994jUa ajqnop e aouabulyUu09 ap nea|qe} uN inod sauuaisakeg SapoyjeW, : Wd }@ 1OUD ‘OUD OS 


anZojeye9 ne K-[00-ZI ,N ‘epeued onbysnes 


0 L Ol 0 I Z 0 SIO9pu] 

8 06 L7I 9¢ Iv Ly LE sed yuoula}qeqoid e19}0A ON 

06 611 L7I 0€ SZ Ee Or jusws[qeqoid B19}0A, 
shapely OROUOd ar ayy 880M) von SH9PUL GL pus ASG) ~ tL ECMO eT — GER he) ee 

419110S3.1}3 Np U0IIIa,| B asinody dIIVU NP UOYII]I,| B ISINndD 

0 p SI p I II L SIO9pu] 

SL 9 191 €L IZ 801 96 sed juouloqeqoid B10}0A 3N 

LS 78 L61 19 €Z Orl raul yusula|qQeqold B19}0 A 
—Wispgpuy ss Auipidzyy a Atouionjuoyy JT Wis Soop] ) Sey N | (RLY. iaoyyqes Yo CY OT Gre ee | 


[eiguas anaindoid np u0yd9]9,[ B as1n0_d 


™ OT 
anb souigul Soy JUSUIA}OVXO OQ JUS[QUIOS SUOTRUISa sas 


‘uepuada, “yuowa}101]dum sasoddns ve ayja,nb ¢ 39 | so[spour 
Sa] snos aseo red snagid synoayjo say sowmssa mod (83861 
‘9961) AuseIS op spoyjsul ek] JasyN suoanod snou ‘snjd 
2q (ZI) ap sajuejsuos wOUd e SIO] soy JURSTTQN US onUI}qGO 
ouuaiss4eq uoneUInsa,[ $9 J{7NON “0 = (49 agnb aydaoxa 
woud & sto] sauIguUI sey estTNN {/7NON 1° (TI) sed guuop 
2 .¢CNON ‘owugui aq ‘9 = |g anb aydaoxe *,/7NON 
aonb uoude sioy sowigur say osIyyn ers NON ® (OT) 
ap sjuepuodai sa] Ins saaxe LoL & SIoy Soy asITUN 5 /7NON 
Jip yuowanny “7 sEepowl a] snos uoude slo, op odA} 
of 9] JuesIHN uaissXeq Mojyeumsa,| j~7NON Issne }10S 
‘JUSWUDAIEdsal ‘aTqeIOUSI UOU 39 BIQuIOUSI JUSTFIUSIS NON 
9 Dy ‘youroanosdsar ““eNON 19 ™ZTNON “IDI 
Jed a[qeiousi uou asuodsal-uou ap ¢ I]Epou J] 19 squioUsI 
uou asuodsi-uou ap Z aJepour a] ‘ayqeious! ssuodai-uou op 
[ Jopour a] snos AJ Np suoyeuMse soy suousIsop sNON, 
"€ 39 [ Sayapour say mod aonb AW] Np suoyneunss 
Sa] suoyuasaid ou snou ‘addj-y1e99 ouIgUI a] onbsaid Issne 
yUO STeU “AJ Np UoyeWINsd,] ap soyooid zasse yUsUOTNAS 
uou jUuOS ased sed snpuaye synoayo sap souusrsgXeq 
suoyjeunss burs say ‘¢ 19 | Sayapour say snos ‘anb 9}e}su0d 
yuede ‘anno uq (7661) 110939 Joye anuUoW jUO] SUTWOD 
sonoweied sap soedsa.] ap ayy ey ims sed juonjis as ou 
AW Op suoyeunse say “¢ 39 [ Sayapoul say snos ‘anb suoj}ON 
"€ JQpOul oJ suep s]qeLIeA anne | op puadap ayja.nb sipue} 
‘Z BQPOU! I] SUBP SUIOUI-I]]9 a[qQeLUeA ek] op puadop onbueur 
g[qeuea oun nb syyIqeqoid ey no sqeious! uou asuodal 
-UOU Op SdTopoOUl sap jUOS ¢ 39 Z SATQ9POU sa] “aoUR]QUIS 
-IVIA PR] Op SdOUdIZJUI Sa] suep solousI aya juoAnod 
jUonbuew soguuop sap sjonbsa] mod seo say 3a presey ne 
alanis juonbueul saouuop soy ‘] sJapoul aj sueq 


Cyl 
Xd yd Tg x eae» 


ay 


a + of + Xa Kg 4% = (hu) Bo, 2 ¢ ofapow 
ia xd + wee yy Kg os 
Mgt Ud Vt q+ %g = (Miw)Boq =z a1epow 


6V 


INIUIIANOS Np UOYdIII,| B ISINOD 


dSq np eco sadepuos sa] inod sagA.1asqo saquUu0g 
Z nesqeL 


Cate Age xe Xo + mg a 

gt d+ XJ + 9 = : T 12POW 
‘yUdAINs Inb a[qeiOUsI UCU 
asuodgl-uoUu ap ¢ }2 Z SA[9POW sa] 39 B[qeious! ssuodsI-uoU 

op | IEepoul a] SUOJOpIsuO. snou ‘UOsTeIedUIOD ek] Mog 
“SHO PUP Sito}e12.P 
jueyodum siquiou un s]dAaI ‘Sadepuos anenb sao ounso nb 
‘Z negjqe} XY] ‘OO,] ep JoWoOsaN 2] 39 snquiMjoD op seul 
3] ‘Teisues Mmamooid J] ‘MdUIOANOS 9] SNIP ONA UD B66] 
DIQUISAOU Op SUOT}DO{a XNe I9}0A op syyIqeqoid ke] ‘anne | 
39 gugjoid yeprpued 9g] We}9 BU039}e9 ouN jUOp sjo_duOoUT 
sanua sqnop ke s0UasUNUOD ap xneo[qe} sop inpold 
quo gSq Np xnei0ja[901d sosepuos sa] ‘AISIOATUA, 91S 
ONC PI op yoreasay ASAING Oj 19}Ud_ J] Jed 96K] US dSTTeOI 
‘OryO,| OP ykig.] Suep ]e10}e19 edepuos no “(qSq) Od 
a1e1¢ aAoyongG np soguUuOp Sap jURAIOS snou Ud AJ Np uoy 
-Pull}Sd,| @ WOUd e stoy ap sadAq buto say ms sagseq souUdIS 
-9heq suoneUuIss say suoreduiod snon ‘siojoid yeprpues 
Ing] }sd Jonb sprogp s1oous sed yuo.u nb stew uous] qeIqG 
-UWSSIPIA JUO0IN}OA INb sId9pUl sIM9}d9]9 SaT Saye yWOS JUOP 
uose} eB] Op JUdANOS puadep je}[NsaI Np spmyyoexa | ‘UONIET9 

oun.p jueuses a] odepuos Jed ampoid ynaa uO,] anbsioT 


(a) 30} 


O14O.,] OP 3874.1 SUBp 
[¥10}99]9 adepuoOs UN & UOHeONdde 9uQ “Eg 


‘adA]} JUSISTA SUTUIOD 


(ana g’z) dxo (mu py'f nied 


(awa gars z)dxa 


= (AWagy iy MMlny 


apessod 4 no Z[ ue — (4)serp] x “NV 
= §o/wi jo (6) op asioaut,| red aguuop ysa (444g) re, 
goueuea eT op g op dWa. iso Mag “4 aseo 
red suoneumss sap a]]9W10}90A uoIssaidxe oun so Ww nO 


ie te 


(pI) PPA “th 


600z uIN[ ‘a}anbus,p senbiuyos 


189 WI op UOTNQsSIp e] op 

uoneurxoidde oun,p doURLIRAOD-90URLIVA op SOLYeUI eT 
‘92S ,Y el op uoye[ndod ey] ap 
gy[tey ef Isa “Ay NO dIFWeNS UOT[QUeYSE UN mod sptod 9] 1s9 
'NIS/IN =" ‘[dutoxa reg ‘J op TNA. B anyeag “Mu 
yso “wy ya ayeNS ,y ke] nod nuuod spiod 2g] 1s9 “nm no 


ul‘) 1=4 
mid me ¢ = (“Ag 
H 


ysa seo ,(/ ‘7) e[ suep souesodsa | ap 
uoneumss,JT “HZ, = 4 anbeys mod ““w oases red 
gouvledso | Ioumysa mod 7 uonyoas e] & SaaNsnyI sajjao0 anb 
SoInpsod01d SoUIQUI So] JUSUID}OVXS SUOAINS snou ‘a}INsUq 


em! 
. CYA, Ys | I] | 
fe zy _ 114i 


g 


- UHliy, | 1eHli 
ZH” tH 


mqyliay 


Inod jaTyouIG ep Uoud e sIo] say suOsT]yN snou ‘Z uOTdeS 
RB] & JOP SayTUMN] SUOTNTOS sap sWia{qold 9] JOWAS.p UV 


ee lI Za) Z 
tei eS =f ehaeatig 2 oe a) + 4g + 
ae ‘i ey by xg mg = (“) Boy 


Jed 9uuop 
so a[qeiousI uoU asuodsl-uoU sp sJa=poml UQ “YY ‘‘Z 
Teaaymeonbeyo, nod, WMC“ 157 — “ar no. (z)..uanb 
uodey ougu ey ap “a. “ar = “ws ose red souessdso,] 
Inod aiesul]-3o] s[apow un MWUuep suoAnod snou ‘so; V 

‘99Ud O[QNOp k Nes][qQe} UN,p a][99 onb ourguI 
RB] JUSUII}DexXd 3S9 (E]) Op souL;quuasIeIA eT ‘Y aes onbeyo 
nog ‘saeys soy mod mayesipul un.p so a] onof saqnus 
g1diy e nesjqe} un mod seastyn fy oatsopoutuLa} e] ‘ou0g 
t='x|way =u lay 4 ='ylg= “uno 


my fi 


I=f 
(4* 1) Boy tL ay Cay ls eee Le 
ip 


I=! 


i! 
(Gea Bape ink 4 (Tela aopl Sa a 4) 
T ON 


> QJUBAINS 
QOULQUIDSTeIA-BO] eT oILI99 suOANOd snou ‘y ayeNs onbeyo 
mod ‘siopy “4 xed sauSissp ang ned uonearasqo 
UOS 19 | B a]e3a 4so “y aJURepUodsaLIOO sayueNbuULUT soguUOp 
OP SOLYROIPUT [qeLIRA eT ‘adAIasqo smo{no} yso ‘YY onbsing 
‘aIHeYS sseuuoj[Hueyse uN suep sayeys sop aya JoAnod 
sou0s9je9 FY Say ‘agArasqo smofno} ang suosoddns 
snou onb (7 ‘*"'‘T = ¥) Y ddIpul,| ap sanbrew asuoda1 op 
J]qeueA sUIIISION Ee] ©Y jos ‘saqua adi e nes[qe} uN sueq 
‘odAy uoneoyneys oun,p ajduiod mua} yUuoUTWIOD aqUOUI 
yins inb uorsuaj}xo ajduns ey] ‘uoNeIspuod op slues 99 
SINpOUI .p ule s1puajs say ap apqissod 4sa |I ‘stoyaynoy, 


ondojeje9 ne X-[00-ZI .N ‘epeued onbuysneis 


‘onbruoydg]a} adepuos un sup sayUaJayjIp uoNdeTas 
ap soujiqeqoid sop e nol] JUeUUOp sma}oR}J saQne.p jo 
soddvis us ost ev op ‘uonRoyneNs e] ap ayduroo 31a} mod 
uoneispuod ap sed yuouuaiduiod au sajapoul 39 apoyjeul sou 
‘apms ajuosoid ey sueq “Yo Jed sanjoajja 919 1UO 966] Ud 
OIYO Ud S9SITP9I XNVIOJOI[9 SOSEpUOS Sa] ‘a}a[dWIOd asAyeUue 
aun suep UOTeJopUOd oUN JOSeSIAUD,P JUSIAUOD [I 39 1e}q Jed 
no uorlsal ed uoneoyyeNs oun e Japso0id ynad uo ‘sodeuoul 
op [e10} s1quIoU Np 9jdexd UONeUIMsd ouN.p asodsIp 
uo] IS ‘ouoYyds]a} op o1suMU UN,p snjd yUepassod saseuoUT 
S93] Inod alessao9u JUSWITeBa jsa sprod up ‘esdeusU 9] suep 
souuosied sp s1quiou 9] Jed salapuod ane yop asuodal ey] 
‘aseUdUI UN SUBP S9MOIIAIOJUT Sd QUUOSIad ouUN Ig ‘souUOSIad 
op UOU jo SaseUDSU Op 9NjITWSUOD jsa onbruoYydsya} UOTMUEYDE 
un ‘ojdwoxs seg ‘sia9puodojne urjd un uojas saynoaxo 
smo{no} sed sa.u WE eB ‘uepuadan (p661 HON : C661 
SPYRIART) SouustssXeq sopoyjout xne sayodde ana jop 
ou UOeOyIpou sunone ‘s1apuodojyne yuoWa}OLNs Iso WO 
red as8epuos J 1S (YO) aloyegye UoNIsoduios ap spoyjaul 
eR] Jed asteol 1S9 OLYO,] OP 3814, OP [e1o}OoTa BBepuOsS aT 

‘youd ® SIO] ap sadAj butd ja soouel 
-odso saqne.p d191URU SUIQW Be] Op IUIJap suoanod snon 


COGET 
BIE cat (Car, | CLA PIG 


Tee plo 
ui 
pio 
aks) 
TI++1 
PIO 77+! CIt+i ¢ cum, ) TIAA -\ Plo 
Ty f= ( a pou | 4) A 
plo 
“QUIQUI 9q] 
TI+4 
PIO ZLI+A TIItA ¢ cupyli T1444 PIO 
fi a = ( a out | 4) cl 
cu Pl 
plo 


Jed sQuuop jso ¥ dBIeUW I OP ONUUODUT UOTEULOJUL | 
sone osed ,(y ‘f 2) ef] mod o[jauuoNIpuos souRIpdsa| “q 
adeya,] WV ‘je[duroour saqua afd e souasuTUOD ap nes[qe} 
un mod uoud B SIO] Soy 19 JA SWIIIWIOZ]e | WuIyap suoanod 
snou ‘juonbgsuod Jeg ‘onbueu afjo,nb ‘7 ja saArasqo 
ysa ayqeuea eT onb onbipur | “777 “177 ‘ZIZ “TI ‘771 “ITI 
= uu mod suigui op ‘ onbuew *y onb steur ‘sagarasqo 
quos “y yo 'y onb onbydun Z[] = uuy ‘agatasqo so 
S2TqeURA SIO Sap suNdeYd onb anbydun [[][ = uu ‘ouog 
(t‘T=u‘my mod u=‘y p w= Y= 'Y ape 
-1S9,9) saatoadsal sayuenbueul souuojod 39 soust] sop 19 (Y 
=‘ty yf = ‘*y 1 = 'y anp-e-3sa,9) asuodal ap sayqeura 
sion sa] nod aseo .(y‘f‘1) eB] ap Jnoayo,, “A ylog aproey 
\so sou {dL e NesTqe} UN e UOTSUa}XO, J “(7 = “yy = 'y 
aIIp-e-}S9,9) ayuenbueul suUOjOS sun Mod “y ya ayUenbueU! 
sus oun mod 'y moeopury yo “(f= “Vv = |x 
aIIp-k-1S9,9) suUOTOD ./ ev] op jo ous] 47 eB] op (/‘7) aseo P| 
ap jnoayja,| mod ruyap isa “A 49 ‘sanua sfqnop ke near) 
un mod sapoyjsur sap gyuasoid suoAe snou ‘1or.nbsne¢ 


uoneoidde dane ja|dwooul a9a4ju9 ajqnop e souabuljU09 ap neajqe} uN unod sauusiseAeg SapOUjyeyy : Wed }@ !OYD ‘!oYUD Sr 


angojeieo ne X-[00-ZI .N ‘epeued enbysnes 


Itty +t yy a sar qo Heating cee 
f=, (FA4)%a (4)! (44) 0 A dk 
icing om Sa coe fer go sg em es 
IT, HRP AREIRH) ie fg ah WE a 
a ‘d= cme? eA ets 0 I] odAL 
wee a= eta aera aaetla eA yadky 
tig IZftg tif IIfg 


(sajguivied ap a1quiou 93] 38a d 3a ‘a9.13U9 


a[qnop & nea{qe} UN suEp sauUO]OD ap ja SaUST ap sarquiou say yUOS ¢ 99 7 ‘AWA, 189 “a ) %Q woLd & sioy ap sadAy burp 


‘uonelnuns red saps sap jo sonbuidura saguuop 
sop juesITQN ud aUAINS UOTDES RT B SoIedUIOD yUOS }9 


[ nea[qe} ne sowmsal yuos oud ke sto] ap sadAy burs sag 
S( fie S (aa 
(ad br deat 
(21) 1 
op hit Sisuge (ih WeERe wine 114 
(5*)- 5 = (44). 5 = Mog = Me 


HINS ]I SUIS TUIZEp 189 (1661) T1091 
330[D op om) ‘uoud e Io] ap addy Jorwap a] ‘adA} Jomueid 
np ised e uoude 10] op odd} owarxnep 3] Mua}qo mod 
We] SUOCAR,] SnoU sUTWOD ([]{) suep Q = 'Q onb quesod 
ua uoude 10] op odd} swaLyenb 9] suosstuljap snoN 
‘g[qerous! ssuoda1 
-UOU dp J9POUI a] Snos onud}qgo jsd UOUde 10] sJeTWOId 
e] onb sipur} ‘ayqeiousi uou asuodal-uoU op a[apoul a] snos 
nuajqo AWA. ap eyooiddez os 39 onurunp “ww ap gWa.] 
anb mod ™“g op uoud e 10] oWaIsION ey suONqLMe snou 
‘quog ‘(uly snjd woud e 10] swainbuto ey issne moa) “1 
» wy mod say] suONNjOS saT[aMUIAE,P IOIIAI,p 
UBe (1661) 1109 39 S8O[D op aJuR\suod LOU k 10] eT IS2_/7/T 
aula} 9] 39 ‘Z‘P = 7‘y mod "wu /""w-d = "A no 


LV 


[ neaiqe 
(chan > MY a calle 
I yy ase 
(11) re) 
<C ew az Ifa 
by (are ay fai 
Me ud 
bi (a at “al OA = 
| 8) fies = 
Ry Ul f 
tlio [ al IA = Mig 
JULIO} 


eR] snos uoude 10] ap odd} SWIdISION a] SUOSsIUTJap snou 
‘AW np suoneumsa soo juesiyyn uq 7 10 ¥ ‘f ‘2 yno} mod 
9 = ™g onb juesod uo [°Z uoNses eB] B ILOEP FLA ouMU 
-os]e o0U op udhoW Ne APL NP suoeUMsa sa] Iua}go 
Issne suoAnod snoN ‘syuepuodsi-uou sap ja syuepuoda 
SOP UOHLULIOJUT,| OP SIOJ eT e JdWIOd JUDUUDN} AJ Np suoyn 


Rusa soo ‘juonbgsuos reg “2 / ett yy _ Cy 49 
Pid 
aan 1/7 17/Z 
x Soe mea Bae tis u = ul 


if [ 
17! a Clty ee qd 


na Wg 1-6 ap uonnjos ey] sa “Gg no 


600Z unl ‘ayanbue,.p sanbiuyoo] 


‘gi = My « pata aa ain, 
¥ ™ (4 oF es ESA Eee ~ 
ee yy }UOS A $2] 
Jejduioout Z x Z neafqe} un suep jt] UONTOS sun apassod 
ouUo[OD op oaieyuowslddns osreur oun puenb ‘ojduioxa 
Ieq ‘SOW suoNNjOs sop osned e sfnu ano JUsrIeuMod 
au suIeus9 NO (766) ‘T0919 Jayxeg op Mw op oyondxo 
suo} e[ Jam suoanod snoy ““w op AWad Mw sed 
suousisep ‘uoud e to] ap adj ougision 2] Uy9p ep UyY 
‘Te10}O9[99I1d aBepuos 
un suep sjuepuods1 sap mmjso onb awau 9] ana sed ou 
ytexmod syuepuodgi-uou sap aiayoid yeprpueo 9] ‘ajduoxa Jeg 
‘g]qeious! uou ssuodal-uoU ap s[apoul uN snos syuepuodsl 
Sop MI99 sp sJoYIp sjuepuodsi-uou sap ssuodai ap pyoid 
3] onb juowlayei9ues asoddns uo] aonb dosed ‘saystjduns doy 
ang Jusreumod syuepuoda soy ms sogxe Loud ke sIoy ap sadAy 
XNOP S3D (P661) UMoI_ 19 eg suep sogst[Hn WoUd e sto] 
xne yinpei as odAj auugrxnep a] onb sipue) (661) Hed suep 
sogst{yn uoud e sto] xne yinpal as adAq Joruaid gf “Q = “**4 
32 (Quenbueur uoQeUnO;UT sues agAIOsqo jUSUIOJOTUA 
yso “y no 'y onb anp-e-jsa,9) saqua ajduns e souasuquoO 
ap negjqe) un,p seo a] sueq ¢ 10 1 3no) Mod g = '!Q 
aonb 9)da0xo adAj Jommaid ne onbyuapi 3sa odAy oulerxnop 
3] ‘SOULIN) sanne p uq ‘uoUd e sio] ap addy Jorwiaid a] suep 
IM) ms (amessaogu 3s2,u oud 8 10] aunone.nb ‘seq snjd 
JOOP 3S9 [I SUMUIOD ‘aIIp-e-jso.9) Loud & Io] sUNONe suUOp 
ou syuepuodal sayz Ins sexe UOUde 10] op od} owarxnop 
Otel =Weei rales 4 mod. od (Aad = 2 AO 


eae (G6 wh ess IZ 
(01) ithi¢ Di ea Ra7e ig AS 
Itt+¢ ++ 
IZ, « (Umea CU SG Il 114 
g 1g ASR ag acy SE 


ica = if 10 | iia | fe | no} nod 
sa syuepuodal say] Ins sexe WOU e 10] ap ody JotUIaId aT 
“syuepuodal 
Sg] Ins ssexe vould e so] suojadde say snou ya '“4 sanuuos 
sasuodal ap synseyjo sep uoNoUO] ua yuswanbrn ”g woud e 
SIO] So] SUOUTULIZ}9p snou 4Ip jusWANnNyY ‘s[qeious! ssuoda1 
-U0U snos nua}go AW.] 2p eyooiddes as jo onutunp “w op 
dNa.] enb axaruew op ”“Q sianqume sed suosuoumu0s snoN 
‘qUeAINS UOSe} ev] op Oud e sto] ap sodAy buts suosodoid 
snou ‘ajuIeQyUOD 3399 snosg ([66]1) [0910 Bd0[D guadsns 
quo | swwi0s ‘d ‘ameguij-3o] sJepoul 2] suep JUeUDAIOIUT 
sanourered op aiquiou ne g[eda yso Q/ 1%"? voude 
SIO] Sap auTWOS Be] anb suosod sno -”“Q sanourerediadAy so] 
JOUTULIZ}9p SUOASP snou ‘PLY SUIUOS]e | JOAoyoR p UY 


L1olid & Slo] Sap UOeIIIedS 7°7 


‘§ Op QING. 9p 90ueuea ey sa J op qIWALI 
B ONJLAD SadAJaSqO SaguUOP sap JURUDAOId UONPULIOJUI | op 


esIoaur | ‘(EQ] aded “pQ0Z) UIqnyY Jo Ways “UTED “ueUTIDH 


angoje}e9 ne X-[00-Z] .N ‘epeued onbnsneis 


JUSQUOUI 9d] SUMWWOD ‘sayuenbuew saguuOp sep yueUaAo Id 
91199 19 SagjusUIsNe ss9uUOp sap jueUaAOId UONRULIOsUT,] 
anus soUusIgyIp eB] B ayesd9 ysa_dode/ (CA | g)/,2 segatesqo 
saguuop sop JueusAold UoHeUOsUT,| anb suojye}suOd snoN 


Cott cotta Cott Cott Cott Cott 
Wi, Celi, Cl, CR Uy Cae rs Ll ly, 
Cott Cott Cott Cott Cott Cott 
Cc re lin, Cog Cy < AGT 
Cotta Cott Cot ty Cott Cott Cott 
Ci, CAG, «tel | Gy | ey 
Cott Cotta Cott Cott Cott Cott — zz 
ere eh Oy Gy RI, “A 
Comey) Cott Cott Cott Cott Cott 
i, aan Come Oru | cei 
Cotta Cott Cott Cott Cott Cott 
GEREN ESSER GAN OTy a5 4 EPG 68 yl da 1 PFF 
10 
Zor Ze) 
TEC gy 0 0 Usa ee 0 0 
eae age 
0 Tey, 0 0 aa ore 0 
ly, Zia 
0 0 Hatayy 0 0 eu, 
Zeta, Let yy, = 17 
Te; 0 0 Tel 0 0 
Leen tay) 
ON | veaqypeerO Os Gree OPO 
ZI) (ae 
0 0 Kacey, 0 0 relay 
dey MARSH) Aer 
Wag >) Te IRS pe 0 0 
Cea Ge Syl Ley] 
Meaprnea tern (na eerrem ball 0 0 
Arey taeeyye ware 
fy yy Tey 0 0 0 
Zi+ly T+ lyy TI yy | atl 
0 0 OD i marin cy 
Zl | Oty Fy, 
0 0 0 Aga Eerste 
Cite Uy (atta 
0 0 0 (GEA ah OA ONES LIN 


>€ = £2 Z =] mod juowoenbiun 
ef © 1 ‘7,G SUONSNT! snou ‘aptoyyip sed yso,u xneigU93 
{ya 1 mod uotsua)xa,, onbsind 3a soedsa.p ayney “1oy 


Pe | 0 0 0 
0  iemeel 0 0 
0 0 ecu HO 
0 0 0 0 


jo 


uonesidde dane ejdwoou! 89.jua ajqnop e souabuljUu0D ap neajqe} uN unod sauuaiseAeg sepoujeyy : Wd }@ !OYD ‘!OYyD OV 


andojejeo ne X-100-Z1 oN ‘epeued onbysneig 


Ct+tyy rr ae Cotte 
[ thin, aa A sep 0 0 0 
IZf+yy Ma) 4 Weltg 
0 | TA Iie selp 0 0 
Z 
Bis 
CIty EWRG 28 aber @ 
0 0 es TG Jee 0 
0 0 0 0 


red soguuop quos g ‘p 10 "x 
aseo red sayyiqeqoid sap a][a110}00A UOTISssoIdxa,| SO Uw NO 


(6) ‘ZaV [ wu — (22) 3e1p] 7 + 
Z (N/ wrt — (ui)3eIp] 'Z - = 


Sede ‘3 
(gM | "aS 30] ,0 


sede ___,dede 
(oe and camace I dive 


suuop (g) “f e yoddei red uoneaAusp aqnop Jeg 
(8) (| DS 80] - 


UL ar J ee al CE sic 


dI1199,$ ynad (¢) 
UOLI9}sod & UONNQUysIp-3o] ke] ‘SIO; ‘UsuTsAtdedsai ‘syuenb 
-UBUL SJI}IOJJO SOP INDJIIA I] 39 SPAIOSQO SJIJIIJJZo Sap INd}99A 
By hpi ee ay pe mod (Aa ZA tly 
ii aa) yon (2 Taiy Ma Gey 1) Eh by 0S 


SV 


"SOATNISUOD 
BO] Np Uous\sod e suoNoUOJ XNOp sHUS SOUdIDIJIP ef 3SO 
3 NO ‘, O[ S 3 GOUASIBAUOD ap aIQ}LIO SUTUIOD SUOSSISIOYD 
SNON ‘JUIOWe WOS 9dUdaSIOAUOD op 319}U0 un.nb 99 e nbsnf 
W 32 g sedejg sao suosg}I snou ‘uyug *_[((,u)seIp] = 4 
ig (Ay — AG rf rf . c 
(yee (Gye — "h) + ue BOT = A yuoug]9,[ apessod (A no 

‘ } J 

wh 1-H Z (Z -4 12) = ind 


: yins 
JI oum0s g op (qq) Wousjsod e uMuUNxeul np moyeul 
-1]S9,] Suoua}go snou “(7p¢E aded ‘7QQZ NSeIsV) so1opuoda1 
SOLIVD SOIPUIOW Sop dALIDII SpOU}OU! eI JURSTITN us ‘ouOg 
"SOAIOSQO JUDSUIDIOTUD JUOS hee ase Jed syioayjo soy puenb 
soayua ayenb e souasuyUOD op neag{qe} UN,p Ied eB onus}go 
gouv[quiasielA ey onb ouuoj ougUI ey] e uoNenbs sy09 


E vpe 

(11) Boy ; URE a 
fet 

(121) Zoy - SAGAS - 
ei 

(*4u) Boy - MACS + 
[oe 

(May Boy MATES = [eg 


JUSTASP 
(L) 30] np OLIA\sod & uONdUOC; ke] op sdueIEdsa,] ‘sIOTY 


6002 uinf ‘a}enbue,p senbiuyoa | 


aa) an Cotte 


Cote C = Pano Se ae i 
17f+ 1zf+ 
1=119 7 = ¥ mod ee oleh 
ee 
ZI+! ZI+! . 
Z =]119 | = ¥ nod Fee eer 


Ita 4 1+ 


114 
t+ a 


1 39 [ = ¥ anod 


A ses | 
ll 


soe JUOS sUOT}RAIOSqo-opnasd 
So] “SoMMeNUOS, sod SNOg.. ead 1. TUT 
certig — UNG INTE _IIF*( suoneArasqo sap sajuepuodsai09 
SO[PUIZIEU SOWIUIOS xNe Sa]eBd JUSIOS sayeUISIeUI SOUTUIOS 
sinaq anb suoneaiesqo-opnasd sao e suosodun snoy “a 
$+ hoa Saas a ane 10 aioe LN TET 4 be iene 
dacs) as Crollt/ cd 4) ieee ce rege Uy 1144 suOTRAIOS 
-qo-opnosd soy juesiyn usa (Z) So] np UoUajsode Io] Pe] 
op souviodsa | suostuxeul snou ‘adeja aye0 VY : PW adnjq 
‘Aut N= Mu Yo 


COnt 
PIO @+t+ (ttt ctf 7) PIO 
hy ai a po ceed 
plo 


1) 


17f+ 
AOR Nac 1+ ¢ 6h 17 7) PIO 
174 A = ( A Sou | Aha 
plo 


Cu! 


ey POraleeG_: (2 «Pty | CIA) Ploy 


TIA Plo 
plo 


red soguuop os say[auUONIpuod saoueIadsa 
sg] “(Z) uonenbs,| suep “**¢ yo I+ ZC soanoadsar 
SO|RUIZIeU SOUIWIOS so] IMS sssuUONIPUOS sayTeIMOUN|NU 
soojegye sajqeuea sap yuos “4 yo 44 “I! onbsing 


Eel 

(1) i ade 9 15 ko Ma ae OR + 
aes 

(' an) Boy - (8 + [10 las | wah een OPE ~ 
(ea 

(HayBoy- Clg + [Cae] APO aN CK + 


1): 
(411) 80] ; (Mg 4+ Leg Ore a persion 


angojeyeo Ne X-100-Z1 oN ‘epeued onbysyeis 


Ut+E yy IK 
“CIN SaTPUTSILUI SOUTLIOS Soq 19 sae ‘sanourered sap sayues 
-noo suoneumsa say yueyses "4 ya 4K <l4K genbueut 
SJNIIJO So] Ins suUdAOUL eT sUOTNOTed snou ‘(g) Jed 9uUOp 
30] Np UoU\sod v Io] eB] op douRIJodsa | JouTULIA}9p Mog 


Cet 

(9) (11) 80] (HAg vi UAV AK Ke ers 
a) 

(241) Soy ('24Q fe aa ERE eh 
Is § 

(= "1) oy (24 in MIRE oe 


fg 
(111) 80] (0 iA Ly RG “ ae) 


SULIOJ B] SNOS dILID9S ynad (¢) WOUa}sod B 
Op ep ep acl Te Rega pee modinde angi 144 
<M saguowSne sajqeueA say Juesiyn uq : 7 adnjq 

‘yuaatns tnb J 39 q Sodeje say juzy0dur09 
(LL6I Wigny 32 prey “Jasduieq) (OWA) esiTe19Ues UONeS 
-TUITxey-s0uelodsy suIQuos|e | JuesTyN ua g snouresed 
3] Ins (¢) Ud agUUOp LIOa\sod B IO] eB] SUOSTUNTXeU SNON, 


Wa euryti0s[e,.J 17 
(dg 


oIIp-e-1S9,9) sanouresed so] JowMNsa mod wy sWyUOS]R | 
oUOp sUOSITYN snou ja axa[dulos zasse ysa (¢) uonenbs,T 


il PEELS or Uthat 
(S) (g- %z)dxa “ |80j- 9% - 
1904 


(d : Ah) : Atha Ze 3. 
ryt 
(a. Az )dxa “¢ Jeo: COOK a 
les 
((g . 7) dxa °C |80 Metg ie 
ryt f 
(g Hy) dya' }g01 Hips aK eg 
! if 
(CE iaydxa"¢ |B0y el ELI + 


ryt ! 
(g - 7) dxa “¢ Jeo UE Ee — 


uonedidde dane jajdwoou! a9sjue ajqnop e souabul}U09 ap neajqe} uN nod sauuaiseAeg SapoUyjeW : Y1ed 38 !OYD ‘!oYyD vr 


angojeyeo ne X-100-Z1 oN ‘epeueg anbnsnes 


(g . ttm axa’ } 80 : age rn 


190 


C > M7) dxo x J 20y ; WAY ~ 


ae 
(q- aA wie O24 42) ma 


: UOUa\sod & UONNQLYSIp-Zo] ev] SUOUa}QO snou ‘(p) 
uoud & IO] eB] 39 SUONRAIBSgO sa] mod ({) ep s[erWoun;NU 
10] B ‘(€) SAY (661 WMS 19 JOISIOJ * p66] UMOI 10 
Weg £ 1661 [0039 B8o[O) syeTwuouN[NUW Io] oun eB sonsnfuos 
yuos sji.nb sored ‘uouasode 10] 9UN,p spoWWOd jo 
ay1orjdxa auLoj oun yuastnpodd jayyouIG op uoLd e sto] saa 
‘77 uoKoas BT B saijtoeds yuos ™g sanourerediadAy say no 


at 

(r) i i ie E “in / LIL] 

qed 
squUop Ja]YoUIG ap uoUd e stoy sop (Ma Hey May «tM y) 
aseo red soyiqeqoid xne suosodum snou ‘(Z) a[2pour a] suep 
AW tp uoyeunmsa,] op oyun] uoNNjos oun JaIIA9.p uy 
‘g sop a]]OU0}90A UOTRJUEsoIdaI eT Isa J 19 aSed Jed d0UeI 
-odsa,J so 7X 7X XJ] SUOISUDUTIP sp UI M9}99A J] NO 


(€) dz = w3so] 


SULIOF BT SNOS }LI999I a9 ynad (Z) 
Ud 9UUOP aregUI]-3O] BQpour 9g] “sIopy “oLOsgyeo (7 “y “f “1) 
RB] & yueUayedde uOoNeAIasgO,] op UOTeTWYe,] Jastoaid mod 
M7 goususdxa.p urd sp mayen [xd 9] suosInponut 
SNON “Joumsa Be (g oIp-e-}s9,9) sanourered op siquiou 
3 d WOg ‘aqeious! uou asuodal-uou saae sjayduiooul 
soussUTU0D op xneaiqe} say JosAyeue nod juowuenbsy 
9ST]N [MNO uN jso oMedUT]-3O] BJO9pOul J] “a[qeUeA 999 op 
uonemls ve] op pusdsp asuodai-uou ve] ‘asuodai ap ayqeuea 
onbeys mod ‘anb puajyue-snos nb 30 “y ya Sy anuaje 'y 
10 'y aNUd UOTLIDIUT,p SoULID} Sap osnvo ke a]qQeIOUSI UOU 
gsuodai-uou op s]9pour UN. p Issne j13e,s |] ‘}o[dwoout saqUA 
g[qnop & s0US3IDAUO0D ap nea{qe} np jUeUdAOId sadAIasgoO 
Sosed op siquiou 9] onb swIgUI 3] JUSUIO}Oexe }sa saNouTeIed 
ap sIquiou 9] onbsind ‘ames jsa amesut]-3o] sJapour 9c) 
‘g]]NU yso syndedsaI 
smousdns soorpul sas op yjonbeay syodui.u ms g suLey 
anbeyo op suluos ey Jo auOsa;V0 (7 ‘y ‘f ‘7) eT nod onpusye 
aseo red (gnoayja no) souonbey ey yso Mu. ay = we no 


(Z) Z‘l=/] 10 ‘TZ ‘T= by epi Weaey iF ast =1 Inod 
ties xX + HG aXe + 
I Z I , 
Ud + Wg t+ A+ % q+ % = (Mw)Zoy 


cv 


ged “y 30 'y “y “y sajqerea say sayno} nod ajqeioust 
uou osuodsi-uou op sjapoul un suUOssIUTjap snoN 
‘(suotsto9id ap snd mod p66] UMOIG 
Jo yreg onb isure ‘ggé6] pale] 30 Joyeg JOA) sonbuossye9 
SO[QLURA Sa] SagIsa}uT yUOS A.nb uOdse} OUIQUI e] Op dITeOUT] 
-B0] s]9poOUl a] SUP JWOUOTIOR] sooIdQ}UI oO yUOANod 
soaneordxa sajqeuea sop ‘uepuedag ‘ounone suosiyynn 
ud,U snou ‘saatyeor{dxa sayqeurea ap sed suosodstp ou snou 
anbsing ‘ollesut]-3o] UOTOUOJ oUN jURSITQN Us sayuoUTpIOd 
sgjqeueaos sop e xu suorar snou ‘uoneumsa,, mod 
sajqiuodsip syeql] ap soisep op siquiou 2] onb sonowresed 
ap snjd yuaUO0S soULTQUIASTeIA Op UONOUOJ 9799 SUTLOD 
“OXxIJ 189 Ly ae Ui ane 
-N ~e tf =*y ¥='y f=*x d= yl =u no 


f 
(1) ayaa]. Fa + (a) Boy - AE + 


! are 
ZI+ap)\Goy . CI " IMa)Soy . AK ms 
I I l 


: QURAINS SOUPTQUIDSICIA 
-30] eB] Muago.p uye seMMouN;NUT IO] oun jUdAINS 
SUONRAIISGO.p SdLI0s9}"89 ston sad onb suosoddns snoy 
“SOIfISSETD UOU sJdOTJo soy mod “**¢ yo ‘saanoadsa1 oust 
ap Jo aUUOJOO ap someyusUIg|ddns sore soy mod '/*¢ ya 
CIM ‘Sorpisse[a JUaUIa}@{duI09 synoayyo say mod |“ suos 
-]Wn snou ‘ouog “y op mmayea 47 eT 10 'Yy op moea ,y¥ 
el “XY ap ouosgeo ,f ef ‘y ap oLiosge9 .7 e] e yUeUayed 
-de ynoayo,, 4 sed suojuasaidar ‘suoneAsosqo,p sau03 
-9]89 SION Sao JonsUSIP op UY ‘SdlfIssejo uoU sydaTJo 
SOP }9 SOIfIsse[D yUOWaT[aIWed syyOaIJo sap ‘salfIsseyo yUOUI 
-19IJUd SJdaJJo sop yuRUdUOD 7 x 7 x (x J VOUNsUTIUOD 
op neojqe} un yinpoid “y yo “y “yx “y sap ajatduioo 
oL9s eT ‘sIopy ‘onbueuwr “y puenb 7 =‘y j9 a9Atasqo jsa 
“y puenb | =‘y ‘ouigur aq ‘onbueur 'y puenb z ='y 
}2 ddAIasqgo 3so 'y puenb | ='y onb Issne suosog ‘danua 
g[qnop ve souasUTUOS op neo][qe} UN suep ‘UdtdANDadsaI 
sou0sjeo ¢ 39 J dOIpur mod yueke Sy ja 'y jos 
‘soanue saydnynu e souasuT}U0S 
op negfqe} un e oysoidde anou suopuajys jo oud e sIo] 
burs suoyiogds snou ‘77 uodes eT & ‘sIng ‘deUA a[qQnop e 
Q0UdSUTNUOS op neo]{qe} UN suep dsuOodsI-uUOU Fe] op sWaTqQo1d 
3] Joproge mod ;wA oumpuosye un suojuasoid snou 
‘T'Z uonses ve] YW ‘jJo[dwoour sanue s[Qnop eB soUasuUTJUOD 
op neojqe} un suep g[qeious! uou dsuodal-uou ey Jaen 
mod souuaisskeq suoneuss bulo ap suoyndosIp snon 


SUdIS9ABQ SIIQPO|L -°Z 
“SUOISNJOUOD SoUTeLIDO SUOJUdSZId snoU ‘¢ UOT}DIS BT YW “SouUDIS 
-9heq suOneUIT}sd sap souRULIO}Iod Pe] JouTUeXxs,p ue ‘omy 


-I9ANOD ap alyIqeqoid ey yUoWIITe39 suOTNoTed snou ‘UOT}S 


6002 uin! ‘aygnbua.p sanbiuyoe, 


91399 sued ‘sjuepuodgi-uou sap jo sjuepuodal sap asuodaz 
9p sjyoid jo soyuenbueul sapuUOp op sasejUsIMOd syuarayJIpP 
INS sogpuo} souUsIssXeq SUONPUISS Sop sIeIq sa] 19 WO 
so] Jareduiod nod uonejnuns red sapme sap e suomodal 
snou ‘p UONDES BT Y “UONDI]9,| Ip S991 syey[Nso1 xne_ nb tsure 
AW np uoneumsa,| souusissAeq SUONPUWT}Sa soy suoreduI09 
SnouU 39 O1YO,] SP 1€14,] SP [e10}09[9 adepuos np jueUdAOId 
sonbuiduwia saguuop ap xnof anenb e suaissXkeq solapoul 
so] suonbrydde snou ‘¢ uonses ey YW ‘aseo red saytjIqeqoid 
So] JowNss mod 9gsiyeigues (WA) uonesturxepy-oouviedsy 
sujuosje un suojusssid jo sowusIgyIp uoude sio] 
buts 90a suatsgheq Sa[apOUl Sap SUOI9pIsUOD snou ‘7 UOT}IaS 
R] Y ‘suonoeas anenb ud ddsIAIp 389 d[OIUe,] ap ayins eT] 

; (77 WONDDS BI IIOA) 
sogudoidde sajqeueaod so] juest{yn ua soddess ua no oy 
-Heys sseuuo]]HUeYIN uN,p SsonssI sagigpuod saguuop sap ke 
WOISUd}Xd 3399 yUOWOTIORJ Jonbydde suoanod snow ‘a,duns 
ySd sognua sojdnjnu e neajqe} un e UOISsUa}xa,77 ja]dWIO0S 
-Wl sanUd sIQnop ke sdUasUTJUOD op neaqe} a] JasAyTeue 
nod xn9,p unoeyo suonbrjdde snou ‘sing ‘stsaput sima}so]9 
sop 39 syuepuods1 sop sioy eT e yUeUdAOId UONeULIOJUL | 
op juapusdsp uoude soy soy yop suatsaXeq soyapouw 
xnop suojuesgid snou ‘ouoq ‘ja]dwoour sanua s[qnop 
B doUdsUTUOD op neo] qe} UN sUep dIJLIDA sed JUOWATBIOUNS 
ysd,u e[90 aonb a]9A01 UONeTNUNS Jed opmed sHou sew 
‘aseo red saoueiedso sa] mod AJ Np uoneumsa,| anb sayyad 
snd ((iOq) souusAour sonbyespenb smoaus sap jusuUOp 
souudIsaAeq Sapou}sul So] ‘ayqesousI UOU dsUOdaI-UOU ap sed 
ua, nb anuoUul UO ST] ‘ya[duIONUT saQUA a]duUIISs e sDUaSUT]UOD 
op nesjqe} un mod suejsuod uoUd e 10] UN 9YsT]N JUO 
(1661) ZINYIg 39 JayUsYIS “uIgny “830 ‘s}uepuods: so] 
Jed smo} UOTeULIOJUT,| Ins yUoWNbiun sagseq sonbundura 
uoud e sioy ap uaAow ne suuaIssXeq ayooidde oun 9gsodoid 
UO (8661) Hed 1° (p66) UMOI 19 eg ‘o[qeJous! uoU 
asuod3i-uou ap souasgid ud AW Np UoNewMss | suep dT] 
UOnNIOS op swa]qold ap aluas 90 JoUMOUOD op UY 

‘d9.HUd BIQNoOp & d0UDBUTIUOD op neo]qe} UN sup dT] 
uonnjos sun NoAe ynad APY Np uoHewnss,] s[onbe, snos 
aMessao09U 39 JUesyJNS UONIpuos sun g0U0UD 3UO (7661) 
199 19 Jaye “(ZOOT PIO * 6661 “110912 WMS) agyuasgud 
29 & AW Op uonewnsa,| op ay] UONNOS ey op onbujow 
-093 uonesdxs,T “(L661 Ssiaqua[oy] 19 S|SIYDI * 8861 
pie] 39 Joyeg) sanuo sajdums e souasuTUO0D op neoiqe} 
UN_.p sed a] Suep sadgsodoud 939 JUO SITU] UONNIOS ke] Ins onjIS 
3S AW Np uoneumss,| soyjanbsa] suep suonipuos sa] 

‘(sagy}iei9p 
snjd suotssnosip sop mod (p66]) UMoIg 39 Ye onb Isure 
‘(Z66[) UelUOoUTISIDNG j9 JosIoquasOYy “‘Iayeg NO fp UOTOOS 
R] MOA) sodAj-spesg spueis op yUSWT[aMIGeY JUD}USSQId 39 
onbrun uonnyos ap oae sed yuaanad ou amesuty]-3o] s[9pour 
np sonourered sop (AJ) d0Ue[QqUIOSIeVIA op UWMUITxeul 
Np suONeUISa So] ‘sed 39 SUR “DDUL[QUIOSIeIA Sp UOTDUOJ 


angojeje9 ne X-[00-ZI .N ‘epeueg onbysyeis 


RP] op yeooy, umMuUNTxeUT UN jUdANOS jUaSsTUIMO}J soyTUNy] 
SUOTINJOS S9d ‘“S9Sed SoUIeYIID sURP a][NU ysa dsuodaI-UOU 
ap si[Iqeqoid ve] onb sumsa jso [I no satu] suoNnyos 
Sop JuowWonbey suuOp AJ] Np uUOHeUIsd | ‘a[qeioUs! UOU 
So asuod9J-uOU PR] IS ‘AoUdsUTJUOD ap xneva{qQe} So] SURG 
‘(p661 UMOIG 19 Weg 
‘ ZL6[ USYD) steiq puesd snjd un.p jo soueUeA pueld snjd 
oUN Pp Sd9YORIUS SUOTeUITSS sop ke Nal] suUOp ssuOodsI-UOU 
RB] 9p UONRIIO Op OWISTUBIIUT 9] JUOWD}9009NUT JaTjIOads op 
no sasuodgi-uoU Sd] 1911899 ,.P We} 9] ‘syuepuodal sap Inja0 ap 
dgJJIp syuepuodgi-uou sop asuodg1 op jyoid 9] 1s “uepued 
“9D ‘(IT o8ed ‘7007 wqny 39 FINIT) souRTquIasTeIA e] ap 
SOUSIZJUT.| SURP SUTUNTTS ang nod dsuodss-uOU FI Op 19]J2,] 
VOW 189 soyuenbuew ssguuop sop uonesio op ouustuessul 
3] onb asoddns uo] I§ ‘ajqeiousi1 uou jsa asuodal-uoU 
OP SWSIULdOU 9] ‘SUOTIPUOD Sod sue ‘s}URpuOodsI-UOU say 19 
sjuepuodai so] mod owigui 3] ang sed ou yeunod syeprpues 
soy Inod saoucigjgid sap jyoid 3] “‘Jononred yeprpues 
un juamndde sayja,nb uatq yeprpues un inod souargyo1d mg] 
sed juonbipur,u yuapuodal inb sauuosiad sa] Is ‘e10j09|99I1d 
asepuos un suep ‘ajdwioxa seg “a]qeious! UOU ssuodaI-uOU 
op sojopour sojadde yuos ZYNIN seo xne jUepuodsau0s 
xnao aonb spur) ‘ajqeious1 asuodal-uou ap sajapou sajodde 
JUOS YYW NO YVOIW seo xne yuapuodsauos nb sajapout 
SY] “Sdd9AJOSqO UOU sINs][eA sop pusdap ssuodss-uou ouN,p 
a]Iqeqoid e] onb aryrusis mb 99 “(wopuns jo Jou BUIssiU 
mod WYYWNI) presey ne sed juenbuewl ou saguUOp sz] 
Jo ‘Sd9AIOSqo sa9uUOp sap yusWIoNbrun puadap asuodsi-uou 
oun .p ay]Iqeqoid e] aonb arzrusis nb 90 “(wopuns jo Buissiw 
mod yZYyW) pryesey ne yuenbuewl sosuuop soy ‘oUIQWI-aq]9 
gjqeuea ey suduios A ‘agipme a[qeueA 3yn0} op ajuepuod 
-9pUI 4Sd jQ19}UT,p a]qeueA oun mod ssuodgi-uoU 9UN.p 
ayiqeqoid ey aonb ayrusis mb 99 “(wopuns jo Ajajajdwoo 
BuIssiM Mod WYOW) presey ne jUswaja;dui09 jyuenbuew 
saguuop sof MHoAes e “({] ased ‘7907 WIQny 19 BNTT) 
sou08gjye9 sion ua saredsl ang yUoAnod (sIdgpul smo} 
-d9]9 Sa] ‘ayuaTeAInbs asgrueUT op ‘No) sasuod9J-uoU sx] 
“SQUUOTNUDUISNS SINSYIOY Sop Je}SUT .] 
® sajuenbuewl suoneAlasqo sop sWIWOS SIDIWIAp S90 9)1ey 
suoae snou ‘sogudoidde sased xne sloapul simojso[9 sop 
uonEaJe | & sde"Is sayoexa snd suonorpoid sop s11u9}qQ0.p 
jue}9 Tedrouid ynoalqo aNON “aTUZAOTS UP 166] AP a19SIqQ9Td 
np uolsess0,] & asttea1 anbrjqnd uormdo,p osepuos 2] mod 
19 7661 ap AOA JouRg UONI]Tq [eIoUSyH YsHUg 2 mod 
J]Epowl uN IMs sagpucj uOHRnduN.p sopoyJsu sop gsI[NN 
JUO (100Z) IMaqoYy3}20H 32 premUDy ‘sysIoqualoy 19 (6661) 
aye 39 JouuTYS ‘WIWS ‘onbiqnd uormdo.p ssepuos 
un suep sld9pul sima}da[9 say Jasse[d mod xneuomou 
xneasal sap ins agseq ayooidde oun g\dope juo ([00Z) 
BUIOTES 19 elOIeH “WIT ‘ejoIaUOyy ‘owe;qoid 99 JoyUOULINS 
op UY ‘SIO9pul sma9}o9]9,p xne} ne yUoWIIIT[eIed sJUSUIsNe 
UONDI]9,| BP SjooJ syey]Nso1 sap uoToIpsid sp mas, ‘(p661) 


uoneoijdde dane }ajdwoou! 99.ju9 Bjqnop e aouabuljUOCO ap Nesjqe} uN nod sauuaiseheq SapoUjeY : Ed }@ !0YD “!OYD ZV 


“Dye vaIOYOysedsA : JouNOD “99409 ‘10L-9E1 


[nogg ‘agJ0D Fk] ap syISIOATUA, ‘onbysHEs ap juoWayIedag ‘mossajoid ‘yIeg BuNsNoA ‘ Z160E VO ‘BIsNINY ‘eIBI0IH Jo ada][0D [eoIpew ‘enbysyeiso1q 
ap juswoyedag ‘inassajoid ‘1oyD UOM lef £ 9910D ‘[OL-9EI [NOYS ‘9910|D kB] IP PIISIOAIUF) ‘SIWIOUODY,P INSU] “sYydJOYSe1 9p INa}delIp “1oY4D sunss-0g ‘| 


ne] 19 (0007) Poe TSW 30 AyjouueLy “Ayjouuey,y guuonusU 
UO, SUMIOD (€86[ JOUUOD,Q 12 SIOAJ\I) SoJUeNnbueul 
saguUOp Sop dUMUOD sasldgpul sosuods1 so] Jaye op 
J]quiayoid jso [I ‘SIOQpUT 1N9}99]9,] 19}0A WreuNOd Jonbo] mod 
JepIpuked I] IMs yUId0e,] OW UO] IS “soploop sonsojowoy 
smo_ onb suoniodoid sowigui soy suep soum xne sed 
JUSPUSI 9S dU SIO9PUT sINd}D9T9 S90 ‘JeINUad Ud Nb JoNUOUT 
ap ule sjeprpued sa] aNUd sI9puUT sIn9\de]9 soy MTed9I JU 
st ayjonbey soide.p 766] Op ouTeoUIoWIe a][onUSpIsoid uON 
-09]9,| Op S99UUOP Sap 39 SHasnyoesseP] NV C86] 21Q0190,p 
[e10}99]9 BsepuOS Np soguUOP Sop d]ULUTUTLIOSIP osATeUR oun 
® apaooid quo (C661) Wry onb Isure (7361) ‘T]O9 19 YoImusJ 
‘onbnuo sed 90 e JUoUUaTedde ¢ UONdES ek] B SaILIOOp 
sonbuidura sapmg sou op SIOl] ‘oyesoj}oo[9 osmod oun 
SUPP 319] Ud s}epIpued xnap soy aud yed9,] eB MaLodns }s9 
diquiou Ina] puenb ynoyns ‘onbyud 3sd slogpuT $1n9}99]9 S39 
op s]O1 a] onb onbipur juswrayeBo e& (S66 |) WITY ‘JooI adeIUI9 
-Inod a] anb 9gaaza snjd yUoWdT]qQe[qUIASIeIA 3S OSepUOS 
un suep dAJasqO sId9puT sinajde[a,p asejUs0InNod a, onb 
yo109s UNMIS op syooidde oun uojas sonuajqo sonbuidurs 
soguuop op udAow ne asquoul e (6/61) Auog ‘sInoayne 
soo Tuueg (SQOT Apouusy jo yosnely “UMRYY ‘ E007 
Auseyg 39 UdyD {S66 WY ' ¢€g6l JOUUOD,.—C 10 SIOAPI 
‘7861 URUIDH 19 Joyoog “ueWlasI\ “YdIMua,T © 6L6T Auad) 
$IN9}03[9_Sdd op adwWiod juRUd) UD UOTDII9,] ap syeyNsal 
Sop uonsIpsid ey] op opmoexa | JoJolowe ynod uo] aonb 
10 [eUIZ 1eI[NSOI 9] IMs aJUeyOdUT sOUEpIOUT oUN ITOAe nod 
SIODPUT $N9199][9 SAP 3JOA ap JUSUWa}IOdWIO9 a] onb g]9A9I JO 


sWIY}WOS]e { o}1WI] UONIOsS 


‘ 99UdsUTUOS Op neajqe} { s[qeioUsI UOU ssuOddJ-UOU ‘ dUUDISgAeq aSA[PUY 


Sopnjd SdsnoIquIOU ap ‘SUTOWTURAN “($66 TBAOYIA 39 WI3}S 
‘UIGNY * p86] WIS) 19}0A 9p syoU UOHUSIUT,p TU yeprpued 
un mod ayeu saousisjoid op sed juajsajrueur ou sjuep 
-uodal sa] Is aplyeA asuodal op dU039;e9 9uN SUWOD sagqIen 
yUOS sasidapul sasuods1 sa] ‘sasepuos suTeyI9d sued 
‘yueuses 
3] opoid mod suaissheq sajagpoul sap Igjuasgid jo yep 
-Ipued onbeyo mod joo1 indde,| sp uoneumsa,| suep saymenb 
-UVUI SUONPAIISGO $90 JdIS9}UI SUOMISOP snoN ‘(sonuuOd 
-UI yUos sasuodgr xnap so] no sed) sa[e0} sasuodgi-uoU 
Op SIQUIOU J] JUSWONbIUN SUUOP SUIIISION LT 39 IQJOA Op II] 
-Iqeqoid e] Ins ss9uUOp sop JUsWONbrun yUanUOD sUTeTXNop 
RI ‘SINd}d9[9 Sap SadUdIgJOId sa] IMs sa9uUOp sop yUuoWoNbruon 
jusnUO0S aTeyUoWIZ|ddns o3rew asad ey ‘(stogput ja ‘sed 
JUDWIDTQRIQUIASTBIA BIDJOA OU “JUDSWIDTQe[QUIDSTeIA ~13}0A) 
IQOA op ayTIqeqoid eT 3sa aHNe,] 19 (SIOgpul no ‘Dy no g ‘VY 
yepIpued) M9}99]9 | sp sousJaJoId eT Sd SaLIOsa\e9 sap oun] 
‘(E007 Auseis 39 UoYD) OFYO,] OP [e10190]9 Bdepuos 9] suep 
‘g{duroxo reg ‘(2]2}0} ssuod9I-uoU dIIp-k-}S9,9) SoTFISse]o WOU 
SJQ99IJO sop 39 (aTJaIed ssuodds-uOU dIIp-k-}S9,9) SoIfISseyO 
quoWaTanred syNnoaYJo sop ‘soIfIsse[o JUOWIAIOTUD ased op 
SJIIOTJO Sop JUITJUOD Jaap 99 ‘sau s[qQnop eB sdUadUTUOD 
op negqe} un suep osepuos op soguUOp sop suIMsol 
UO,] IS “(866 Jodnoz 39 SdAOID { 666] 1990H 9) Buousne 
xne} uos onb amsoul e xnou9s owig{qoid un jUDdIAOp 
gsuodal-uou ke] ‘sasepuos sap yednyd ey suep ajuonbe17 


uopoNnpoHnUy “| 


‘Wa 
: $9J9 SIO 


‘9[QeIOUSI UOU dsUOd9I-UOU dp d[2POUI UN snos sauUsIssheq SUOeUII}Sa build ap s9}9Udod say Joredwi09 
Inod uonenurs red sopnja sap e suopgo0id sno ‘suoNse]9 sa] Inod suoneUINss sap s}ey[Nse1 say Jo\aIdJ9jUT.p jo JosI[yN,p 
JUSTAUOD [I JUIWTWOD JoNSNI]I nod o1yO,| Op 3€14,| SUP B66] Ud SasITed1 xNeI0}99[9 SadepuUOs sop URUDAOId saguUOp op xnof 
ayenb suosi[yN snoN ‘aiWI] UONN[Os UN snua}qQo 3s9,nb stoy anbeyo AJ| NP suoeUMsea so] onb 39 somougyue OU k sIO] 
SION Sa] onb syej[Ns91 SINd{[IOW op yUoUUOP LIOLId ke SIO] So][aANoU xnop so] aonb sipue} ‘samnoug\ue sapmje sap Juarenbrpur.nb 
99 & JUDWAIIVNUOS ‘AW Np suoneWINss soy aonb sg}9udoid sormojjiow op yuowonbyewg3shs sed yuo.u voUde sio] 
SdIQIWIAId SION Sop opre,| B SONUD}QO SoUUdIS9ARQ SUOTBUITSS Sd] “SIOPPUT SINDd}09|9 Sap Ja s}Juepuodg1 sap asuoda: ap sjyoid 
Sd] PU DOUSIDIJIp B| 9p o}dwI09 319} op ue segsodoid oud e sto] Sa]J2ANOU xNap jUOS sane xnap So] 32 Ja;dwIooUT s9.QUS 
[duis e nes[qe} uN.p sed 9] Suep sagsi[IN 919 UO LOLI & SIO] $90 ap SIO], ‘9[qQeJOUsI UCU asuod9I-UOU op s[9poul UN snos 
aseo Jed sayermounjnur sayiqeqoid so] Jawiyso Inod yajyoug ep woud e sto] butio suoredwios snon ‘jo[youIG ep uoUd e 
SIO] SAP 39 OIILDUT]-3O| 9[9POU! UN JURSITIIN UD Jo]duIODUI sg.QUA B[qnOp eB sdUaSUTQUOD Op NeaTqe} UN sUEp (AJA) 2OUR|QUIOSTeIA 
op WNWIXEW NP UONLUINSI,| SP SIITWII] SUOTIN[OS sep dWIE[qQo1d 9] sIPNosgi Inod auuaIssAeq aspoyjou oun suonbrjdde snoy 


guinssy 


, W4¥q SUNSNOX 39 LOYD UOM lee ‘IOYD sunos-0g 


O1YO.] OP 3¥IA.] OP XNV.10}99]9 Sasepuos xnv UOHVddde daA¥ ja[dwIOdUT 
99.1}U9 3[QNOp & IdJUISUTUOD op Nea{qu} UN AnOd sauUaIsadeq SapoyjdW 


Lv 


anBojeyeo ne X-L00-Z1 .N ‘epeued anbysneis 
9S-Lb ‘dd ‘| .N ‘SE “IOAN 
6002 un! ‘ayenbus,p sanbiuysa, 


"€8/-8LL ‘UONRIDOSsy [eoNsHeS ULUOWY 
‘spoyjay Yyoavasay Aaaung uo uoljsas ayy fo sa8uipaaoig 
‘AOAING YYeaH uoyejndog yeuoneN oy) JO} UOTeUIISa 
soueeA densioog (6661) ‘dL TIT 1 H ‘Joe “qd ‘oo, 


“LIE-S6Z ‘p9 ‘“Maiaay JDIYSIDIG jOUOYDUAJUT “UOIeNTeAd 
uy ‘suljdues Jaysnjo Japun sojqe} Aem-0M} UO adUapuadapuT 
Jo sisal (9661) “WD ‘sHeqoy 19 ‘OV ‘YsuIg “Yd ‘seuOY] 


‘9¢9-0€9 
‘TQ ‘UOIDIIOSSP JvINSIDIG uDI4aUp ay) fo joUInor “BuI;duIeS 
Ja\snjo JopuN soysye}s J-JO-ssoupoos o]durts 10} Jamod pure [AI] 
jo suosueduios oduies-[jewisg (2861) SNL ‘ory 3° “Wa ‘sewouL 


"PSI-SEI ‘OZ ‘sousunig fo joudnor upIppuny ay] eyep 
AdAINS JO} spoyjow deys}oog saIy) SuUedWOoD (7661) “UY “Jouls 


‘BelIa A-Josulids : 10 X 
“MON “do4jsjooq ays puv afiuyyovl ay] (S661) ‘Cl ‘WL 32 “f ‘oeys 


“uy ‘SUOS 2 AZT 
UYOL : YOA-MON ‘suUOIIDAdasqQ aivlavauyn “(H861) “AWD “1992S 


‘oyngsu] 
g[suely, Yyoessoy : ON “Wed asueuyl yoessoy 0'6 asvajas 
‘Jonubu aspnsunj NKYYGCNS ‘(pOOZ) BWaNsSUl s[sueU YoIeasoy 


angojejed nev X-[00-Z] ,N ‘epeueg onbnsneis 


‘OUT ‘SUS 2 ATI UYOL 
 WOA-MON “CUUTYS [°-D 32 SIoquIeYD “TY ‘sp_) ‘vip Aaauns 
fo sisdjoup ‘ayepdn pue jesteidde uy :sAoains xa[dwios woy eyep 
asuodsal [ed03a}e9 Jo sisAyeuy ‘(€00Z) “U'C ‘SewWloUL 19 “SN ‘pf “ory 


"pET-STTZ ‘81 ‘ajanbua p sanbiuysal, “saxa,duios 
sajgnbua xne sajqeorjdde aseuuoj[yueyosel ap sapoyjow sa] Ins 
syuaoa1 xneaey sonbjangd (7661) ‘end 1° CAO ‘NM OWN 'f Soey 


“IPT-LET “€8 “uouDIZ0Ssy 
JDoONSIIDIG upoiuaup ays fo jouanor “eyep AdAIns xa{dwios 


WM soussgjur Suydwesoy (8861) FLO TM “ANY ‘ory 


‘OET-17Z “OL “uoUDIDOSsy JooUsIIDIS 
upridaulp ays {oO pouAnor “sajqey AeM-OM} Ul BOUDpUddopuUT puUk IY 
JO Ssoupoosd JO} s}sa} porenbs-1yD :sAoains ajduies xojdwios woy 
eyep [eoLodajye9 Jo sisAyeue sUT (1861) LV HOS 19 “WNT ‘ory 


‘90€7 
-667Z ‘UONRIDOSsY JeosHeIS URLIOUTY ‘ssuiaayy JDINSIIDIS juior 
ay fo ssuipaaz0ig ‘AdAINS UOneWWIT AWANOY pue uonedionied 
uevIpeues 24} 0} poyjow uoeWwNss soueeA dejs}ooqg 
ay} Jo uoneordde uy (¢00Z) “gy ‘edese] 19 “q “oYoney “YY “ojsueT 


“Sp-SZ ‘uowig|ddns ‘g] ‘sousyojg fo pouanor 
UbIPDUDD AY] “Sayeullysa ADAINS Ul SJOMID SINSROW 0} SPOU}OU! 
Joyo pue densjoog (8861) fA TM 1° WN Oey “OL Teaoy 


9}enbus,p SasuuOp sap ins sasaujOdAy,.p $}sa} Sa] unod enbiyeid desjs}oog epoyujewW su : 1990g Je JUOWNe|g Ov 


angojeieo ne X-[00-ZI .N ‘epeueg onbysnes 


‘6L77-SLTZ ‘61 
‘sousojg fo spouup ayy ‘sojdures paynens 10} spoyjow uonest[dor 
poyoda1 poouryeq pue ajruyyorf ‘uonezueoul] jo samodoid 
gjduies odse] oy} uo aJ0U VY (1661) I ‘Preqnein 19 “TY ‘Woy 


‘OLT-OLT ‘Ph ‘UDIDUSIIDIS UDILUAWUp AY “SOSTRIS J 'UOLIOJUOg 
jo osq. seep Aaains xoajduioo YIM syUsINYJo09 uorssoidol 
jo Sunsa} snosurynuig (0661) Ta “preqnein 1 “Tq “Woy 


“LLG 
-Z[Z ‘uoneIOOssy JeoNsyRIS URdLIOWY ‘spoyjapy youpasay Aaaing 
uo uolrag ayy fo ssuipaac041g ‘des}oog dy} BuIsn saposida asnqe 


3p JO UOHeUIsa soURLIeA (p661) ‘CW ‘AXsporg 19 “TV ‘soysny 


‘Z9OL-LSL ‘Ly ‘solgjawmoig ‘3uyso} sisoyyodAy 
deysjooq Joy sourjopind OM] (1661) “WS “UOsTIM 1 “d ‘eH 


‘pLI-OLI ‘UonRIoossy 
jeousyeis urouiowy ‘spoyjayy youwasay Aaaing uo uoljIag 
ay fo s8uipaaz0ig “eyep AIAINS YM UOISSIIBI1 INSISO] JO} I-Jo 
-ssoupoos Sunsay, (L661) ‘Cd ‘eunwIpI] 39 “Ty “Woy “Tq “preqneiy 

‘179-6729 ‘88 ‘UONIDIDOSSP [DIISIIDIG uDIIUAaULy 
ay) fo jousnor ‘suiajqoid UuolssoIdeI 0} d0UdIOJOI sJe[NoTed 


YIM SONSHRIS 3S9} ONeIpenb [eoIsse[o Jo asn oy] :eep AdAIns 
xa]duioo yyia Bunsay sisoppodAH “(€661) “Ta “Woy 39 “Tg ‘preqneiy 


"897-197 “SL “UO’UDIDOSSY [DINSIDIG UDIL4aULy 
ayj fo jousnor ‘so|dures o8e\sy[NWI poyyeys uo poseq WwW jo 
ssoupoos pue sduapuadapul Jo sjso} ayeutxolddy (0861) ‘d'I ‘1321/24 


‘LSI-Sbl ‘08 ‘uouDII0Ssp JoISIDIS uDIIAaUp ay) fo jOUANOL 
‘sajdures xa[duioo 10} 3S9} posenbs-1yo poyruppyoel V (S861) AY ‘Atl 


TRH 2 ueuideys : JOA MON 
‘dp.ajs}oog ay} 0} uoyonpoul uy “(€661) “fA ‘TueNYysqiy 19 “g ‘UoyY 


‘T8E-9LE “LB 


‘UOYDIIOSSP [DIYSIDIS udIUaUPp ay) fo joudnor ‘Burjduies KaAins 
Ul sIojeUSS uOHeIqYe) (7661) “AO ‘Tepules 30 “D-f ‘oyftaod 


“Ouy “suOS 
29 ATA UYOL : FO A-MON “C9UUTYS “[°-- 39 SoquIeYD “TY spy) 
‘pyop Aaains fo sisdjoup “siayourered JOpoul SuIjeUWIs 10} spoyj}our 


poseq-[opoul pue paseq-usisaq “(€00Z) “U'D ‘sHaqoy 19 “y'q ‘“epurg 


‘00€ 
“687 “LS “A SUES ‘ja1D0g jvousiDIS jodoy ays fo joudnor ‘SuNsay 
g[dnjnur 0} yoeoidde [njiomod pue yjeonoeid y :aye1 AIaAoosIp 
asyey om Surjonuod (S661) “A “BioqyooH 32 “A ‘Turuefuog 


orydeasolqig 


1=5 


Pes" Ka — SH" 9D 1m ‘s)v},(dH-"dH) 


= (JH : M's) 3 

suoua}qo 

snou ‘7 osomoddy.] 19 (7V) op mued e 4uanbasuoo 
Teg “PHoq!] op gISop un jUepassod sayuepuadaput aized-1yy 
somojegye sotqeuea sap juos ’75 soy 19 GHZH)(_V) = V 
ap saidoid smojea sg] quos G “ ‘T= 5b mod “’y no 


6€ 


=) 


(Zv) Cu’ x —,,— (SH-“"dH),-V,(SH- “SED 4 


suoua}qo snou “({°V) uonenbs | 
12 (Ops ased ‘pe6] sagas ‘ajduioxea ied) sonbneipenb 
SOULIOJ Sop IMs plepuR}s jeyNsol uN juRsTyyN ug 


((v) ‘GHXH‘0)N<—a,— (dH - “dH 4 | 
anb yUaUIa{1I9eJ SUOAOA snou ‘] asaujodAy,| op yueyeg 


J }¥}[Nso1 Np sanaig 


oxouuy 


“Q[OTUE.] OP PUL] B] Jo s[eI9UES oytTenb 
P] Jololowie p stuuiod yUO snoU SUOISSNdSIpP jd SoTTe}UDUTUIOS 
$90 snoy ‘jyolns 9s] IMS SajUSSdIOQJUT SUOISSNOSIP 39 Sole} 
-USWIUIOD sind] op ‘epeueD onbysneis op ‘ounpoje’y] saa X 30 
Jopulg pak enb sure ‘uojapreD gUSIOATU/]] Op ‘OEY NL 
Issne® SUOTOIOWoI SNON “SolTeJUSUIUIOS SMe] op SIMeojyeUuruUexs 
XNOP 39 SIOOSSE INDJIePSI I] JUSUISIQOUIS SUOTOIOWIOI SNON] 


SJUIUIIIIAIUIIY 


"Q]e10} oURLIRA eT Op urTd ne jd aJ9pou ne sanp sayues 
-odurod say yUoINaTe1 nb s[qeasI[saU UOU jsa sseUUOTIN 
-ueyos,p uonoey ke] Is soudoidde densj}ooq spiod sop 
IO10QLI9,P INA Ud d1AINSINOd as JUSAIOP xNRAR] SIT “a[qeas 
-I]39U SO d[9pOUl ne onp (T['¢) a]e}0} soUBLIeA eT op omyed 
eR] onb opos op s[qeosljdau 4so a[eqoys oseuUOT[WUeYO,p 
uonoey ey Is aonb ¢ asoyjodAy,] yUoJsnes ou uejd a] ms 
SOpuo} prepur}s deys}00q spiod say ‘¢ UOdeS ke] B QUUOT]UDU 
elop SUOAR,] SNOU dUIIOD “UTJUq ‘IMapuojod usd snyd saIpmMe 
ong Wop uonsenb ayag ‘deysj}ooq js} np opmnoexa.p 
NedAIU I JoJOIawIe Mod san aye-jnoed jusre1os urd 9] 
SNOS JUSWIOU IWIDISION J] Issne yUa}9TJ01 Inb deys}oo0q sprod 
SIT “§ 19 p ‘¢ deQs}ooqg sasoyjodAy sou omejsnes e sed sap 
pednyd ey] suep omyyns yeraop tnb 90 ‘ueyd a] snos oseuuo] 
-[HueYd9,.p INdUd,| op syusWIOUT sIoTWIaId xnap soa] JUD}oTFOI 
19-xnao ‘usWaTToMIgeH “dens}ooq spiod sap uononysuos 
RB] Op Issne stew 4s} op onbysnejs e] ap xIoyo np yUsWIaTNas 
uou juopusdsp spoujour onou op sajoudoid soy] 

‘a[durts 93801 39 deys}oog a,qnop 9] a}IA9 
}S9} op spoyjour ajou ‘de.ysj}oog sinpgooid ap sed yuarginbai 
au nb ssegnbijduios surow sanbysnejs sap ins s9xe jUe}9 
uq ‘sajsAyeue soy Mod spoumuos aya sed ou op yesonbsu 
sIvUl ‘JSd} op sampsdoid samaiow ap suajgo.p ang 
-jnod jremouuiod [yj ‘ayentut densj}ooq onbijde1 anbeyo mod 
dens}oo0q sonbijdei ap a[quiasua anne uN.p uoNeeIO eT jIeI 
-o}Issoo09gu des}oog a]qQnop uy) ‘({°S) Suep suTWOS deys}00q 
spoyjow ey Jed oumsa js ar) =A, Is deys}ooq a[qnop 


6002 ulnf ‘ajgnbua.p sanbiuyoo) 


un juepusdas jresiayoduros jag (p's) HOdS-oRY ap aqja0 
anb a[2} Joatd yuouronbyo}dwiAse onbysyeys sun deys}ooq 
3] Jed J9}1eN op JasesiAud jUsWIaTe39 suoLuMod sno 

‘deys}00qg s0ueIJUOD ap saT[eA 
-IQUI,.P UONONSUOD ek] & JUSWIIIOeJ sa}depe og JUSUATeB9 
yrenmod sept, J ‘sonbysyejs saqne.p ja saxaydwioo snyjd 
SOJQ9POW SoNNe.p sed oJ suep syn ang yWeLMod suosodoid 
snou onb deys}ooqg apoyjow ej onb orpuajyua og-jnod 
JUISSIL] JO SJUBISBIMODUD JUOS S}eI[NSdI SUOG Sad ‘aTIeOUT] 
UOISSOISOI Op I[9pour uN.p 9}x9}U0D 9s] SUep sjeI[NsaI 
suog op suuop aije,nb uoyejnums red opms oun.p uaXoul 
ne gNUOUI sUOAR snON ‘sonbisseyd sjarorso1d sop yuesI{nn 
uo sonbiydde | ap saysAyeue say mod a[19ej 3ysa ]I,.nb uarq Is 
‘Q[9pOU UN INS SagpUO}j }S9} ap sanbisseyo sanbysyes sap Ins 
oindde.s spoujsul aNON ‘sodepuos sap mnjao anb sourewop 
soyne.p suep osonbijdde ang juowalesa yeunod mb 
ajgnbus.p saguuop sop imped e sasoujodAy.p 3so} ap aydurtis 
32 g[eI9Ua3 deNs}oog spoyjoul oun gsodoid suoae snoN 


UOISSNISIP 39 INIVUIWIOS =“ 


“ 


gE 67 9°¢ r'V gsodoid deysj}oog 
9°¢€ ce S‘p oP 1UOLIgJuOg 
g°¢€ Ge g'€ 9'P HOIg-oey 
I'v Ge Sv 9'P PIEAA 
S‘Ol €6 L‘0 a agigpuod salen 
g°€ ee OOO! 0001 a919puod uou salen 
as9L PSL GsIL PseL 2poyiz 
0='n 0 = 'o 
AIVUA °H aIVUA 9H 
JHCULIOJUT WON JHeWIOUT TVNOANN AWAHOS 


TWNOAND AWAHOS 
%S ep uUuoHKIyIUsIS ap Ines ne jafar op xney 


g neaqeL 


snos 


‘JHeuLIOJU sed so,u sdeUUO][HULYDI | 
onb ulepI9d JUSS qeUUOSTRI JS9 UO] IS J[QVIA }OS d9J9puod 
uou sAreu spoyjow ev] onbionb ‘aouessind ey no neoAlu 


ondojejye9 nev X-[00-Z1 .N ‘epeued onbysneis 


9] SWIs0U09 Inb 99 Ud SasnaLIas saouRSI]JNsuI sap JoyUasaid 
yusAnod soaleu sopoyjow say ‘amd aiooua jso pyeA\ op 
SpoyjoUl v] 19 s[eJoqt] doy stored jsa stew “ast{Wn e aydurts 
So TUOLIgJUOg op spoyjow! ey ‘audoidde jaro1so] un e sadoe 
JUO SIMoeSI[YN Soy IS SAUDE DUUOG dUN jSa 09S-oRY 
op spoyjoW vy ‘souessind ap apylad arava] oun ,p suseduiosoe 
919 e@ inb 99 YneULOJUT UOU a8eUUO]]HURYOS,p sOLIeUZOS 
Sd] SNOS SMoyeAIssuOD snjd nod un s}eyNsoI sop suUOp 
SIOJOJNO} & II[q NOOS-ory sp spoyjous ey Jed said ap atams 
‘NBOAIU 9] OUIIZDUOD Inb 99 Ud aIMaT]IowW eT Wea suosodoid 
snou onb deys}ooq spomjoul ke] ‘a]quiosua., sueq 
‘QWUSWUSNe UO]]HURYDE _p 
gyie} ey puenb sjyueyodun sulow jusuUaIAap sapoyjsul 
So] oud syes9 soy onb juoNUOUI sjyey[NSoI say “%S 
& SMOLIDJUI JUSWIIIIBI] JUSWDATIATJO JUSTeY9 INb sayqey|quias 
yofai op xne} sop yinpold yO soAreu sopoyjew say] onb sayne 
SOPOYJOUI So] Sajno} ‘suoTpusye A snou snou sUTUIOD ‘sa}e.yS 
So] oQUd s[eSoUI UONIedgI snos JHeUUOJUI UOU onb IsuIe 
jJueuuoyUT oseuUOT[HUeYd9,, mod ¢ neajqe} ne souasoid 
area yso °F puenb syeynsoi say nuajqo suOAe snouU ‘suOn 
-IpUOd S390 sURC ‘AaBeUUO][HULYSS Pp UONOeIJ BIqQIeJ} SUIOUI PT 
JUSUSIOQI[Ip ITUS}UTeU op sTULIad e snou nb 99 ‘QQ T II!e} 
ap suoTTHURYIP YOO | PUUONSeI9S 19 000 DOI ep UOHe|ndod 
Op d]Ie} UN sIPISUOD SUOAR SNON “INo}euTUIeXxe un ssodoid 
B] dUMWIOD ‘sayeUIsLIO sUOTIpUuOD say suep onb sopuris 
snjd sto} xIp uol[WUeYDd,p salle} sap ddAe sUOTe[NUIIS 
SOUTELOD 9INDIX9 JUSUIITeB9 SUOAR SNOU ‘jsa} ap SopoujsU 
So] IMs suo]]HUeYDI spueIs op jo}jJo,] JoIpmMa mog 
‘sone 
so] onb ajuessind sutoul JUsWIOIEs9g] 1Ss9 19 JHeULIOJUT UOU 
OLIBUZOS 3D SNOS DOLLAIBSUOD JUOUIDIESI] a}S01 s9sodod 
apoyjaul ey ‘uOTeNUNIS Jed apna 399 suep TUOLIaJUOg 
ap 19 WONS-ovY op sapoyjsur say sed assedims ou ay]o,nb usiq 
s}¥\[NsaI suog ap suUOp (aaJapuod uUOU ja s919puod suOIS 
-I9A) dAIeU apoujsUT ev] ‘nAgid sWTUIOD “yUOWarIe[Nde}OOds 
Joloyip sed yua;quias ou sapoysUl soy “nes[qe} 99 sued 


JueULOJUT vou aseuUOT[HURYD9 deAe TWNOA AWAHOS 
OLIBUZDS J] SNOS sje}]NSaI sa] yUONUOD gp NesqQe] I] 


CPAs oss Sore CES OOL Ol pee ome gsodoid densioog 
Sigs Mosse Icy OUST IL en ) 88L PS LS tuoLIayuog 
SESUME GML Gr MGS 9S TES aI MOS ELS Noog-ovy 
O'SSs uMGisseRKEsrinp.O 9or MEO leUN-S's 1 comes iNnINE'S PIEA\ 
bil Sep aS BSa Rh Lk hameel Pine SChgow til) pw Shae ncO's apippuod oAleN 
LE als Pe rd Ee ary A Ee fg (BE apgippuod wou oAleN 
TI99L SAL USL eL UseL SL Ts9L sal pou 
SL‘0= ‘0 os‘0 = 'D sz‘ = ‘0 o='n 
ASSAVA°H ASSAVA°H ASSNVA°CH = AIVYA OH 


JOULLOJUL UOU aseUuUOT[QULYIA 


Iwn0d AWNAHOS 


jJueUOjUI uOU 


aseuuoluueYys? 19 TWAOA AINAHOS SNos % ¢ ep UOHedyIUSIS ap [Inas ne jafas ap xney 


a}enbuse,p seaguuOp sap ins saseujodAy,p s}s9} Sa] unod onbijeud desjsjoog spoyjewW auN : 990g }@ juUOWNe|g 


dp neaiqeL 


8e 


andojereo ne X-100-Z1 aN ‘epeued onbysnes 


p'9¢ 7°89 y°97 S8E 76 Pl arGit £9 gsodoid dens}oog 
OSL STL 47 6tr SOI SOL EL OL WOH} tO 
STL sare ce 6 Chu wei cl sol 79 9°L WOdS-ory 
TOL STL 9'tp 6th SLI SOI 98 9°L PIEM 
718 8°€8 0S 89S O€Z Sy7 7 Sich vil agigpuod oAleN 


cioneroceip me lyoponee GecMipgTTS Gec’cT DEP wr agippuod uou aAleN 

SIL SL ZsAL PseL Ms9L pseL seL psa apouy 
sL‘o= 'D 0s‘o= ‘9 sz‘o= 'D = 'n 

ASSAVA°H ASSOVA°H ASSOVA°H IVA OH 


JUVULIOJUI UOU aseUUOT[QUeYIT 


TvNOaNN ANAHOS 


joeUulojyur uoU 


aseuuoTnueYys? 39 TVAOANAD AWAHOS SNos % ¢ ap uonKdyIusis ap [Inas ne jafai ap xne |] 


Vp nes[qey 
6 66 666 9°68 96 OTE CTp is ec gsodoid dens}oog 
8°66 6 66 9°88 LOBOS OEP GoIg9 8°¢ UOLQ}UOg 
6 66 6 66 v'06 L€6seu lee EP as 8°S HOIg-ory 
O00! 666 €°76 LSE Git HS LE O CE, eae 8's PIEM 
y'18 0°L6 781 s9s €0 €°9 00 10 ogigpuod oaleN 
L’6L 7 L6 S91 gg = €0 L‘9 00 10 agigpuod uou salen 
a9L BSL BL PSL GIL PsL GsxL seal epo—poW 
SL‘0 = ‘0 os‘o= 'D $79 = '0 o= '» 
aSSAVA OH ASSAVACH ASSNVA °H aIVUA 9H # 
JUVULIOJUT eseUUOTTHUURYIA Tvn0d AWNAHOS 
jJOeUioyul 
aseuuoTuuEYs 19 TWAOA AWAHOS snos x% ¢ op UOTBIIUZIS ap [Ines ne jafa1 op xney 
ae neoqey 
0°16 8°76 L6S TOL oh61 P6C 9 iL gsodoid densjoog 
6°16 1'€6 8°89 SLL OIE 60€ II 0°8 TUOLO}UOg 
8°16 1°€6 RIS i We 60€ 89 0'8 HOIS-0ry 
0°56 1'€6 hy del Wa Ba ais 6 OGaeesl 08 PIPEM 
9°8L €°06 OEE 08S 9b Srl . v0 fies agigpuod aaleN 
OOO! OOO! OOO! 886 O00! €S8 OOO! SLE agigpuod uou salen 
a9L PIL BIL PSL BSL PsxL Ts9L  TISeL spouwyW 
sL‘o = ‘0 os‘0 = 'o sz‘0 = ‘0 o='o 
aSSA VA OH ASSNAVACH ASSNVA OH aIVUA °H 


JOLULIOJUI aseUUOT]HUEYI| 


TVNOANN ANAHOS 


yoeurs0yur 


asvUuUoT[QULYsI 19 TWAOANN AWAHOS snos % ¢ ap uOneLIYIUSIS ap [Ines ne jafa1 ap xney 


“a[qeidaooe SUIOWIUBAU STRUT ‘SoJUdLINIUOD SdIOTUIEp 
xnop sodd op ayja0 aonb ajqrey snyd jyUaWIa1a39] stoyayno} 
so souessind eg yooS-ory op jo TUOLIAJUOg ap sopoyjeur 
so] onb omoayjiow nod un jsa jo pyeAA ep spoujour ey onb 
S}RINSOI SMI]IOW Ip DUUOP I]q “SOLMRAIBSUOD JUDUIDINB9] 
SUIQUI SO 39 DOIOpUOd UOU sAIeU SPOUT eB] B VOUaLMOUOD 
yey sgsodoid spoyjoul ey ‘nesAtu a] asWId0U00 Inb 90 ug 
‘aIeBIA ySO Fy puenb % g‘7Z] op jofol op xnv} uN dade aTeIOQI] 
don justAop sorgpuod ajueueA ey] onb spur} ‘tor syeqnsar 
suoq op suUOp sdl9puod uoU dsArIeU ospoyjoUl ek] ‘aIpUuaye 
As yweyey |r eum0D “TWNOANN AWAHOS oueugos 
3] Inod Wp neajqe} Np s}e[NsoI sap Joynostp sed suoduout 
-Wi0o “‘nesAnoU 9q JHeULOJUT UOU sseUUO][HUeYDI,| op 
S¥9 9] SULP S}e}[NsoI So] JUOUUOP gH 19 Wh xnNeo]qQu} Sa] 


Le 


Ve nes[qeL 


‘aIeqA So °F puenb »% 67 op jofer op xne} un 
sade ‘sasodoid dens}ooq spoyjoul ey] e.nb Isute ‘Woo9-ory 
op 19 TUOLIaJUOg op sSaj[s0 & JUDWdAneIedWIOD s[eIOqQI] doy 
JUDUIIIITI] BIONUD HOS PBA Op spojour ve] onbronb “(assneyz 
°F) aouessind ey 39 (areIA FF) NeaATU 9] DUIaODU0D nb 
90 Ud Sa[qeIeduIOS JUOS SapoyJoU sayNe soy sano] “sIqIey 
JUSUISWEI}X9 souessind oun Jed jinpen os nb 99 ‘aoLNRAIOS 
-U0d doy JUSWIOANTUTZEp jSd DAILU OpOJOW eT ‘sed 90 sUep 
SWIQW[ “[QIe} Zasse jso asepuos ap sprod sop ayI]IqQeLeA Pe] 
onbsind ‘sajqe{quias s}ey[nsa1 sap yUasInpoid sAreu opoujsUt 
R] Op solopuod uou ja salapuod suUOISIOA soy ‘IOY Hew 
~1oyur aseuUOTTHURYOY,| Op seo a] sueP TYNOA ANWAHOS 
OLIBVUZOS 9] SNOS sje]NSOI say JUaNUOD gE NeROIGR} IT 

‘sasodoid deys}ooq spoujoul ev] onb ajuesstnd 
snd nod un jsa y09S-orY op spoyjou ve] Uepusdasy ‘oIeIA 
so °F puenb »% 79 ap jueje jofol op xne} 9] ‘neaATU np 


600z unl ‘ayanbus,p sanbiuyoe | 


UOHRIOIOWIe s19S9] DUN SWIQUI dQ-ynad daAe WO9S-ORY ap 
‘sonbijduios snd steur ‘saanoida apoyjour ey e a[qeredui0s 
so oasodoid dens}oog spoyjsut eT % 9‘9 ap joafal ap 
xne} UN d0AR JT[NU ssayJodAY,| Snos TuOUayUOg op 39 Pye 
IP SOPOUJOU! So] Ins JUSWIOANKOTJIUDIS oIOdUIA | 09S-oOPY 
op Ipoyjoul eT] ‘plepuLjs TUOLIaJUOg ep spoyjou vy onb 
9AgI9 snd jofa1 op xney un JUsWIanbyewgshs ynpod aya 
Jeo ‘o[qn sed pleas ou spoyJoW op aruad 90 ‘UONR[NUIIS Jed 
apme ajuasaid ve] sueq (S661) SIOquoOH 30 TuIWelusg sed 
d910QPI9 9][99 anb a{[9} ‘sosOTjaWIe TUOUAyUOg epoyjew oun 
suOIseSIAUS SnoU onb sJassns & Md}eUIWIeXd UL) “SOLNRAIIS 
-u0d (udWIaNbHo}duiAse) oya,p UonRndsl eB] e TUOLaJUOg 
op spoyjew eI Jeo Yueudidms nad un jsd jye}[NSdI 9d) “IIA 
ys °F puenb »% p‘T] ap yofer ap xne} un dane ‘a{eIaqt] don 
a109Ud WOS a]o,nb udtq ‘uOTeNjIS eT sIOToWe TUOUaUOg 
op ojduis opoyjour ey ‘areia yso FF puenb % g‘c] 
ap juris jofor ap xne} oy ‘ayeraqt] doy 4so prea, op ospoyjoUl 
eR] ‘sopoyjou sayqne xne jusWIaANeIeduI0d souvssind op 
yudua}OU onbueur jd sOLYeAIOSUOS do 4sd sd19puod UOISIOA 
ey ‘ued anne.q ‘ayjnu ssoyjodAy,] snos %¢ e Inatiedns 
UIO] Op 4Sd Jalal ap xne} UOS Jed ‘oTeIADQqI] doy JUSWIDANTUTJap 
ySd do1apuod uOU UOISIOA Pe] ‘ued oun.q ‘aseuUOT[WURYIE ,_p 
urd np UONeULIOJUT, | JUOWIaTQeUdAUOD Sed yUd}IO[dx9_,U SaqTa 
IED SdIOOIPSU $}e}[NS91 Sap QUUOP UO SdATeU Sapoy}SW xnop 
S97 “TVNOUNN AWAHOS Inod v_ neayqe) np syeqnso 
So] Jouturexa Jed suosuswmod ‘snou smod jo19juI pues 
snjd un ajuasoid inb ‘fneuuojuI sseuUOT]NHUeYI9,| ap sed 
J] SUeP s}E][NSOI so] JUOUUDTWUOS gE 19 VE xneo[qQr} SOT 
“QIDTUIDIC B] JULUIDIUOD $}e}[NS9I Sd] INs UOISsNosIp 
aou suoloxe snou ‘uenbgsuos Ieg [LSA ssemoddy,] 
Inod aonb ZISqL anu ssamoddy,, smod sjuessaisjur 
snjd jo syueddey snyd yuos si] ‘qr 3° Vr ‘AE “VE xneo]qQe} So] 
suep soide-10 soyuasoid yuos syeq[Nsal saT (% $ Op UOnLOy 
-IUSIS op [Inds UN jUeSITQN Ud) saUUOTOETIS sUOTTHURYDS 
000 § Sa] Ins osejus0mod ua jolol op xne} 2] g[NoIeO 
SUOAR SnoU ‘apoy}sUT 39 aT[NU asoyjodAy ‘uoNTeds1 op ou 
-pusds ‘UOTROTFWeNS op oUeUINs ‘UONe[Ndod anbeys Mog 


UOHBINUIS ap S}eI[NSIYyY $9 


‘Id1 Soiteyuaurg]ddns sye}[nso1 sao sed suoyndsip 
au ja suojuassid ou snou ‘deys}00q sapoyjou soy IMs 
UONLYUSOUOS 9JOU 39 UOTAIBSGO 37399 9UUOP jURIY ‘([°S) 
JURSI[IN Ud SNUd}qO xNdd & S3{qu[qUIIS Zasse STRUT ‘SMoLIayUI 
JUSUIDIOSI] JUSIEJD OSLIVOUT] DOURUIVA OP INd}eUIT}SI,| SOAP 
snua}qo jafo1 op xne sa] ‘([°¢) dens}oog soueueA op IMoyeUl 
-1]S9,] Op Nd] ne tuOUayuog sp 39 WOdS-orY op ‘prem op 
SOPOYISUI Sd] SUP 9SLIVOUT] DOULA Op INd}eUINSd,| JUBSITHN 
ud uoneinus Jed apnya,] ayndexa suoAe snou ‘snd oq 

"Q UOISIOA 
‘SVS anbnysye}s Jarorsoy a] suep sourueIZoid ysa agsodod 
spoyjsul eT ‘NVYWdNS ep useXsow ne snuajqgo JuoWles9 
}So (7°) tuouaJUOg op onbysye}s ey] “6 UOISIOA ‘NVVdNS 


ongoyej29 nv X-[00-ZI1 .N ‘epeued onbysneis 


onbysyeys [oIs1I3o] Np prepurys saos say yUOS OdS-oVY 
op 19 P[k ep J sonbnysnejs soy onb sipur, ‘Sys Ja1orso] 
Np psJepurys dIVOS ke] JUOS dATeU dpoyjoU ey Mod syeyNsoI 
SIT ‘pS UONDIS eT e ALWOSP YOoS-oey ap J spomjoul yy 
12 €'¢ UOTAS ET B BILONP P[LA\ OP J Spomppur BY *7'¢ UOTSaS 
RT & SLOSp UOLaUOg op spoyjoul xe] ‘[°¢ UOTOeS Fe] e 
9LIN9p (sa19puod 39 sorapuod UOU SUOISIOA) dATeU OpOoUI 
re] ‘oagsodoid densjoog spoujsul ek] : sayuRAINS sapoujeul 
burs saz o9Ae saguUONUdUISNs sal[nu sasayjodky xnap 
S23] _9}S9} SUOAR SnoU ‘sUUONDeI9S UOT[HUeYDE onbeyo nog 


389} 9p SOPOyVP p'9 


"S99IPNYD }S9} OP S9POYJOW SasIOAIp 
sop souessind vj Jonjeag mod sagsiyyn yuos suoyeindod 
SOISIWIOP SION say ‘suonendod saynne say mod sassnevy 
yuos soyja.nb sipuey ‘g = 0 oaae onuajyqo uoneindod ey 
Inod sarelA xnap sayno} yUos sasaujodAy sao anb suojoN 


1) =e et = weed - He ICLSAL 
WN Ses Pa SEALE cep 


: SQJUBAINS Say[Nu sasayodAy xnap so] J9}S9} op adestaus 
SUOAR SNON ‘a}URISUCD sOURLIRA ap 39 aI[NU suUUaXOUI 
ap oMojeg]e INaua,p suLIa} uN ysd 73 no ‘solapuod soLeo 
SOIPUIOU Sap spoy}sU PT yURsIPHN ua *3 + "x 'g Fig + | 
= "€ amesul] ojgpour af gisnfe yo ‘7 “ ‘T= mod 

‘uowlanne a} fet 


% = YA Is I 


SOOLNLOIPUT S2[QULIPA Sd] TUJap SUOAR 
snou ‘Juawlgsiogid snjg ‘souRLIeA op asAyeur,p s[apoul uN.p 
yUBAIAS snou ua “4 ap UONOUOJ oUuN,p sULIOJ snos “4 9ST] 
-9pOUl SUOAR snou ‘sUUOTIeTaS UOTTHUeYDS onbeyd Mog 


say[nu saseyjodAH ¢°9 


a]qtey 189 (QOT/T) eseuuoT]N 
-ueyoe p ajeqo[s uonovy ey onbsind ‘ajqejydas0e sa mb 
30 ‘QJOPOU! Np djI]IqeLIeA eT BOUBI sTeUT ‘(aSTUIAI DAR OSeU 
-uo][Hueyoe .p asayjodAy,| @ onp ueyd ne yoddes sed souen 
-PA PB] OP UOTLUITSAIMs d19S9T DUN JdAv) BSRUUOT[NHULYIE ,_p 
ueyd ne aol] ayyIqeueA e] ap d}dWIOd JUST] spoyJSU sNAD “Y 
aes e] Suep aseuUOT|HUeYSE,p sayTuN “u sop mmyed e sade 
[ — ‘u oaae aynens ofdurs amojyeaye aseuuo]]Queyss red 
QSIUIdI DdAR QUUOTIOAIIS 9}0 & deNs}oog UOT[HUeYDS onbeyo 
‘gpoyjour oy00 op uoneoydde onou sueq ‘sonne sqUs 
‘(Z661) ‘T]09 39 ORY IWD9p WO, | SUID ‘aguUO|[HURYDS 9yTuN 
anbeyo inod uvyd 2] Ins sopuoy deys}o0q sprod gg srnoyeo 
SUOAR SNOU ‘sUUOTOITIS UOT[HUeYSS onbeys mod ‘uyuq 
‘Y pun] sp Uoros[9s 
ap auyiqegoid ve] op oSIoAUT.| JuR}a oUTUIOD ‘mM oSepuos 
ap sptod a] juowaduis suosstuyjap snou onb suojoN 


ajenbua,.p seauuop sap ins sasaujodAu,p $}s9} sa] nod anbiyeid desjs}oog spoujaw sun : 990g Je JUOWNe|g els 


anZo[e}e9 ne X-[00-ZI N ‘epeued onbnsneis 


WOOT ANAHOS Oueuss 9] snos snuajgo xno onb sajq 
-euea snjd dnooneaq ju0s TYQOANN AWAHOS vonned 
-91 B] op jueynsor spiod say ‘aseuuoT[NUeYde,p splod sp 
SJUdIAIJIP Sa} Sa[quiasua xnoap juasInpoid souBUgOS sa_ 


JyeUUOsUy 


TWNOT ANAHOS snod uoyHuvyog,p soyre L 
Z nea[qeL 


: OJUBAINS 
uodsey eB] 9p ayes onbeyo e a10a0}Je Iso gI1UN,p dIqUIOU SUIQUI 
I “TVNO ANAHOS PUSIsgp ‘oLBUZOS puodeS 9] SNOg 


Ey 
> 
3 
By 
e 


TVNOANN AWAHOS snod uopHueysa,p sayte 
] neaiqeL 


: ULAINS UOSe} eT 
op SaeNS sa] anus soredas JUOs saguUOT|HULYIE SoITUN OT S2] 
“TVNOUNN ANAHOS Cueugos 2] snog ‘soyens xne uonTed 
-O1 dp SJOUNSIP SOURUOS xNap dSITWN SUOAR snou ‘aQno uy 

‘uonendod ap 
SHUN YOO Z UOITAUDS JUDTUOS SOARULIOJUT UOU sayeNs buts 
sop sunovyo onb sipur) ‘uonejndod ap saytun QQ [| UomAUS 
JUITJUOD SOANLULIOJUI Sa}eQs XIp sap ouNdeYyD “Z op sau03 
-9]89 xXNdp SI] 39 4 Ap SoLIos9}e9 bulo say anus jUsWIadnosdal 
Jed saymmsuos yuos mb sayens xIp suUOp ej99 ‘JyeWLOyUI 
SUS of SUEC| SIUOWIoIE 67 =" 2-19) OO. Is “fy 2 
: NURAINS UOSR] RI Op Z a]qQeUeA ke] 99D SUOAR snoU ‘y 
uoyejndod ap ayrun onbeyo mog ([°9) suep “bo anojyegye 
MaUsa,p sua} np puadsp nb z onbuosa}e9 sjqeuea anne 
OUN.P 19 A J[QRLIRA PL] OP S9SIOIO UOTJBOIJISse[D Jed sarujop 
yUOS soyeNs so] YJNeuUUOJUT sseuUO][HURYOS,| Op sed ZI 
sueq ‘yjney snjd sorupop a sqeuea ek] ap sau039}e9 buro xne 
JUOWI9}9ex9 JUSpUOdsaOD sajyeNs soy ‘JHeWUOJUI UOU oseU 
-UOT[HURYS9 | 9p Svd oJ sue ‘JHeULIOJUT UOU onb IsuTe “HeUT 
-IOJUI ISeUUOT[HURYOI,| JO[NWIISs B SOUNSOP S}UdIIJIP uoTed 
-IJQVNS Ip SOMBUZOS XNOpP SNOS dSIWIdI sURS QO] IIe} op 
so[duns somoyegye suoyHUeYI YOO S aUUOTIDaTaS SUOAR sNOoU 
‘soguuoljuousns suoneyndod onenb sap sunoryd mog 


dv.ijs}00q sprod sap ja suoy{yURYd9 sap UOeAID «—7'°9 


$1019 0S‘0 ‘SZ‘0 ‘0 = ‘0 SmMaeA Sa] QIapIsUOD SUOAR 
SNON ‘4 jo 4 aQUd UONRIAaI0D eR] ajonUOS Inb ‘'1 ap x10yo 


Ge 


ag] ied onb juosajjIp ou saa9gi0 suoAe snou onb suonendod 
anenb soy “¢€=9 8 OT= "0 (I ON ~ "db nO 


(19) “00 +( = J'm eB 


é 
(I+ 7) 
uojas dg910 a] B 
ajuepuodap ajqeura ey ‘¢ be axG aja Be Inb ‘"4 ap sati0da}e9 
ap aiquiou af sa yf no ‘F/T = (2 = “4)d giiqeqoid 
Bl cane “7 “7 ST = 7 nod 4 = “4 onb aya} ¥ uonendod 
ap eauun onbeyo mod ‘4 onbuos9je0 sqeueA oun 
JUSUTUIepUadapUI 9919 SUOAR SNOU “IZDUSUWOD INOg “sg}TUN 

000 01= N ep suoneindod anenb 9919 suoARe snoN 


suoneindod sap uonevety = 1°9 


‘$9 UOT]OIS FP] B UOTR[NUIIS 
R] Op s}eynsa1 say suojuasaid jo pg UONDeS eB] B SoONTeAD 
SOPOYIOU Sa] SUOALIDOP ‘Eg UOT}OOS kB] B 99}S9} ANE WOp 
nb anu asoujodAy,] suosstuljap snou sing “oWsANdedsa1 
‘suo]]queyse sop ja suonerindod sap uonersio ev] SUOALID9p 
snou ‘79 12 ['9 SuOoes xnYy ‘jJNHeUuOJUI uoU onb sure 
JHEULOJU sBeUUOT[HURYOS UN ,p sed I] SUPP SdgUUONUSUISNS 
}S2} Op SapoyjJoW sop souessind er] 39 NedAIU 9] JOUTWIeXO 
Iod uoyenuns sed opma sun smooyo suoae snon 


uoyenuns ted opnjyyq*9 


oF UOTNQLYSIp eB] ap oInatIadns ononb x] e soredurod js9 
(I's) J noog-ory op J onbysnes eT “(0 + 1)/O = O 
yo soidoid smajeA sod op UOeLUIeA ap JWUOdTOYJI00 3 
yoo ("ga [C%g) Sa] sosttespuad sueyd ap siayyo.p 
soLneul ey ap soidoid sinayea sop suudAoul ke] Jso Y ‘ayduuts 
aroyegie aseuuo]]Hueyos,p ueyd un snos ““g ap QOURLIeAOS 
-OOURURA Op SOLBU BP] Op IMayeuNtyso uN 1s9 C™D Sa no 


“dH GHC") ABD, - “dD 


BOGE De 
I 


Jed atutyap isa onbysiyeys eT “(p007Z 
aMYSuU ssueuy yoressay) NVVANS Jelosoid a] suep 
99]NI9X9 JS9 d]]9 SUID ‘aIIVIUT] UOISS9IS9I eT Inod sinayne 
S90 Op dpoyjJoUl eT op UOHRIdepe oun suOSTTyN sNON ‘a}HOqT] 
op soidap op siquiou a] mod ayrtemypayeS ap uoIdaI09 
RB] Ins s9puo}j }sa Inb “(1 86) HOS J9 OFY Op sIpPIO dWWIOTXNap 
ap sgjsnfe aueo-1yy onbysyeis el ap (€QOZ SeWIOY] 19 ORY 
TOA) J UOISIOA 9UN IAST]NN & d}SISUOD spoy}eUT ayNe ou) 


(p's) “(9 - 


= (3's) 4 


}HOIS-O8Y 9P HY 9POWIAW P'S 


‘(POO AININSUT a[sueL 
yoreasoy) NWVdNAS Joordoid np udaXsow ne a9yndexa 
\so ampgooid ayag 1°? 7 uonnquisip e] ap ornouiadns 
ananb e[ B (9's) ,7 anbysyes ey suoreduioo snoy 


600z uInf ‘ajanbua,.p sanbiuyoo, 


eRe 
baOrey 


= (9's) J 


(es) (@ - “dH, CH C's) “AWD,@ - “dD 


SULIO} eT snos 
(0861) '821[24 gsodoid e.] suTWIOD MayeuTWIOUSpP Ne aisle 
QUoqI| Op SoIsap op sIQqUIOU SOAR PIepUrS PTeA\ Op deo 
-Iyy onbyysyejs e] op J UOIsI9A oun IUIJap suoAnod snoy 


PIFAA PP A PPOWVW E'S 


‘QsT]NN a[apow np uoTesiAsurered vy] ap [e1ouas 
ua puadap spoujsul ay199 aonb suojON ‘sayeNs ap aiquiou 9] 
SUIOUI saguUOT[HURYSE BseUUOT[HULYDE .p sorted sayrun,p 
SIqUuIOU Ne [e3a \so p dLOQI] op salsap op aiquiou a] “0 
ap nal] ne G/ 9SIAOI UONKOIIUSIS op finas un soae ? by 


UONNGLYSIp ve] op smouadns ononb ej ve ‘GO ‘ ‘[ = b mod 
‘(’o%s), 7 anbysneys spues3 snjd ej suoreduros sno 
C'S) vogd enbUSHES ap Id e| N 
b SM du, b 
(Z's) Tai ree ee CC 
9- “g7H) 


sagigpuod sanbysnejis GO say suojnoyes ‘ayinsuq 
‘2 op quougia ,b a °o red 39 YY ap ously .b By Hy 
Jed suojuasaido1 ‘ampgooid 3999 oIlID9p Inog ‘enburdus 
opnjd MI] suep s}ey[NsaI suo op WeuUOp ai]9,nNb syuOUW yuo 
smoayne sao ja Jonbydde e ajdums 3sa ay]q (0661) PreqneiH 


yo Woy Jed sgIpma a9 & MOLaUOg ap spoyjsul ey] 
TUOL1IJUOG IP 9pOuWIAL 7S 


‘yuessind jusws]qeuuosiel 
19 opryea ‘ojdums js} un ke yWNoge ‘aseuUOTTHUeYDE,p 
ued np ajduioo sed jus ou mb ‘agigpuod uou uOISI3A 
eR] Juewuoju sed jso,u oseuUoTHUeYSS,] Is onb suojON 
‘sorgpuod UOISIOA eT ynopMs ‘onbyeid usa juoUNWanbsy 
ssgsT]yn sulowmuesu jUOS saya ‘HeUUOJUT aseuUOTTWUeYDE 
Snos s}e][NsoI suog dsp JUdUUOP sapoyjeuI xnap sad onb 
39 & aIpusne,s sed assind au uo.nb uaig “7 uonnquysip 
R] op smMoauadns ononb e] e sareduioo nesanou ap jsa nb “(3 
£*] ‘s)g onbysnejs ej ms amndde,s aaigpuod wou uoIsIoA 
eq “7 uonnqmusip e] ap amouedns ononb ey] e sereduioo 
yso mb (Zp) red aguuop (a ‘ *m ‘s) 7 anbysyeys ey JasipyN 
B dJSISUOD SATLU SPOUJSU ev] Op saIapuod UOISIDA eT 


SOAIVU SOPOYJIWI xNnIq [Ss 


"YOdS-0RY 19 pPyeM ‘TUOLIaUOg 
op sopoyjsul saz Imod issne stew ‘suosodoid snou onb 
dens}oog apoyjowl ej Inod jusuajnas uoU asinba1 SUOp 3s9 
¢ asampodAy,| ap aytpryea ey] aonb Jousi]NOs ap yUITAUOD |] 


gq 


"gr A 


Vv 


angojejye9 ne X-[00-ZI] .N ‘epeued onbnysnejs 


de.ns}ooq 
SOURLIVA OP INd}JeWISA,] 9SI]QN suOAe snou ‘g OTIS 
B] op uonemnuns red apn. sueq “(°"g)““A op (9) “A 
d j2 W Snos UdSIZAUOD IMa}yeUsSa un s9]no} juaqIssao9u 
‘yuowoAnoedsal p¢ 18 EG ‘ZS SUOTNOSS xNe saxLIOOp 
‘O9S-oRY op 19 P[eAA OP ‘TUOMAJUOG op sapoyjeul so] 
‘g UOT]OS FP] B UONLINUMIS Ied 
apnjd SUN SUEP SdONI[VAD Sd}NO} JUOIAS SoT[q WOUayUOg 49 
PIEM ‘09S-oRY op sopoyjsur sop sanbiyioads suoneorjdde 
sop onb sure ‘onbneid ua sogsijyn siojred yuos inb 
SOAILU SOPOYJOW XNap IIOARS kB ‘pf UOT]DS B] B aSOdxo aITeaUT] 
UOISSOIB9I LI SP 9}X9}U0D d] SULP jS9} Sp SapoyJeu sauTe}IIO 
JUSUISAQHG SUOALIO9P snou “‘UoNDes ajuesgid eT Y 


IIIVIUT] UOISSIASIA 
B] Inod sapoyjgul Sa.gne souIey.ay °*S 


‘Inojesipyn | red siptoads 
SVS soguuop ap a[quiasua uN suep sdapueUap sonbysnyr}s 
soo Japresaanes inod « = VWAONVLSAL LAdLNO SdO » 
uononysut,, onb tsute ‘ayinpoid jos sayeynos anbysye}s 
e] onb Jopuewop mod aresssogu 3so [SA] wononnsut | 
‘snjd 9q “SVS oP OAY Wnpsooid ey yuestpHn usa sonbrydde 
yso oosodoid deysj}ooq 389} sp opoyjsul ev] Is so1apuod 
aenbysyejs oun muajqo imod amessasou ye}9 LTHOTAM 
uoyonysul | onb guuonuew efap suoAe snoN : ¢ anbinway 


‘TRIA JOJJO Ud JS Nb 39 
‘C'dH ¢ 4. ‘S) 7 Jasin e@ neamnbs (Q ¢ {mM ‘s)’7 dens 
-J00q onbysye}s eT op uonesiyN,| onb sonbiydun jusy;quies 
suoneaAlasgo sad “A’x = (“x|%a)“q ounoy ej snos g*x 
= ("x | 4) “q amesuty sJapoul aoU amI099I suoAnod snou 
‘oxy yso ““g ~onb a1ooua yuesoddns ug ““g—g = A no 
‘Q=AH: °H Yuoweaneuayye ‘no “gy = du :°H 9yinu 
asoujodAy,| 19}s9} eB sdUSEp anbysneys aun sumU0d 3aja1d 
-IaUI ag ouop ynad ("GH * 4M ‘S) J dens}oog onbysneys 
ey] ‘deys}oog uornqmysip e] snos o}I]ea1 US sed 9] 3S9 INb 39 
‘oxy ayyuenb oun surui0o “™g jyuRyIen Us onuUD}qQo a9 nod 
(“gH 5 ,'M ‘s) 7 = (0 £m 8)2 siTeB9.1 ap yjounoymn 
uoneoydxe sug ‘auUOSaTIS UOTTHUeYOS onbeyo nod “YH 
OP SINI[VA SI] JUSUIDTTONUCU JOUS IIOASP Op II}TAI,P stutiod 
B snou (Q ‘,.M ‘s)°y ap uonestyn | anbsind ‘uoneynuns 
red apme anou ap seo 2] suep Isure 939 B Ud I] ‘spororsoid 
SUILLI9O Op Seo 9] SULP apowUOD snjd sropred sa uoNdo 
aiatwiald ey ap uOnesTNN,] ‘aleoul] UOTssoide1 op s]9pour 
un,p was as uo Is (GH! MS) JZ NO (Qf, M ‘S)% 
yuest[n ua sanbydde ane jned deys}oog spoyjout ayou onb 
suos ap ("gH : ,.M ‘S)7 = (0 ‘mM ‘S)"7 aonb Januoul 
ap aloyjip sed yso.u ]] “s 3 y onbeys mod “(g ‘9A 
‘s)z suep “g’x- 4 = ‘a sed 74 juedejdwar ua (9 
og MS)? dens}oog anbysneys ej suosstuyaq : 7 anbupway 


9}9nbus,p seauuop sap uns sasaujodAy,p s}s9} Sa] unod enbiyeid desjs}oog spoyujew su : 1990g ja JUOWNe|g ve 


angoyeyeo nv X-[00-ZI1 N ‘epeued anbysneis 


CO" GH5,,M ‘S) J ap [noTeo Np anjoxe jos 
au UOneAIasSgO suNONe nb uye ‘s 5 y mod ‘ aM de.ns}00q 
spiod anbeyo & (,, 01 x] a[dutexe sed) oantsod moyen 
ainad say oun Jaynole e aystsuod oula|qold 99 JayIA9,.p UdAOUT 
uq ‘aesodoid deys}ooq jsa} ap spomjour ey sonbidde.p 
JUBAR JUSUID}NaI0N agysnfe1 ysa sjaroisoid sao op uskou 
ne agnoyeo deys}oog onbysneis ey onb ‘oanesou ey] suep 
qo gsjun ysa 4 — u onb samsse,s mney qT (C" GH fg, ‘S) f 
dens}ooq onbysyejs ev] Jaynojeo mod «4—u ap nal ne 
4— ju Jasin yuaanad sfarorsoid saz “9 < aM satjanbso] 
mod s 5 y sgylun.p siquiou ne [eda ysa ‘, u “dens}oog 
onbyjdo1 .g ek] op asATeuR.| SUP SogsI[NN sUOTeAISqO._p 
aiquiou a] ‘Sys onb sya} sjarordsoid sureyis0 sueq ‘(766I 
‘]]09 32 OVY ‘ajduaxa red ‘MoA) s 3 y soqtUN souTeyII9 nod 
[NU JUDIUA|GLIQUIASTRIA BIOS ,’M anb sud}ON : | anbuoway 


"dH red g0vduias jso.9j9 4’ Jed gov[duiarysa “m onb 
adaoxa ‘uawaanoedsal (pp) 19 (¢'p) (Zp) suep “9 10 "Fe 
‘(9 $°m ‘s) 7 B a[qeredw0d oJ9TUBUI op juuuENGo,s ag 
re) nd anb tsure “C"dH  ,M ‘S) 7 demsjooq anbysneys 
eT (pp) © (ep) (Zp) suep T= "x=H=4=6 
puenb (p'Z) & npr es (Zp) 180} op onbysneys eT (Zp) 
suep yieredsip (4—u)/(4— N) IMaory ap ‘u = MOY 
anb alatueul ap sauu0ayoeel jU0s sptod sa] Is ‘neaanou 9q 
‘aTJOYO9,] & stu yUOS sprod say yUOp UOode} eT ap sed puadap 
ou 19 ZosomjodAY,] WeJsyes ayo,.nb suojoN ‘s > ¥ mod 
‘Q < % onb uonlpuos e ‘Sys ap OAY snpeooid vy] suep 
LHOIAM uononysur,| JuesWN ua onuaj}qo ang eLMOd 
(9 {*m ‘s)g onbysneis ev (Zp) suep ‘ajduioxe seg 


cama Sh 
(bb) tase Tio 8 8 
zl qix — 4m mR ; 
13 
: yy °>* Shae Rs eae _ ™ 
(¢"p) xm & _(ixtxta re g 
no 
coy {(4 — u)/(4- N)} "OO 


v 


(9 gu) (4 a pebenn a) _ “gay 


=1(0 fo mos) y 
4S “(€°Z) OP SUN RT SNOS dILID9,S 
ynod tnb ‘([‘p) ap salapuod uoyHUeYoS,p UOISI9A 9UA 
(lp) suep T= "X= H=4=6 
puenb (7'Z) B wnper as Iq ‘ay[nu asamodAy,] snos “" ©7 
uoHnqsip ey yns (9 */)7 onbysneis ey “([p) sueq 
pas! eH 
AAG OP goipn f 


i) 


VO 1y nk ual xx sink) 2 ag 


ce 


no 


"90 


v 


4) 
0- "dn (a ("x") H),0 - “dw 


=(@:f)} 
ajuvinos onbysiye}s ey Jostjyn suotzmod 
snou ‘aiajua uoneindod ey JaArlasqo suolAnod snou 
If 394 9H :'H oAnewaye ssomodhy,] anuoo 9 = gH 
:°H «ayynu asamoddy,] 10}s9} suomsap snou onb suojed 
-dey “y gun] mod soyuepusdspur juoulanesuty sopqeuea 
ap 4 UOISUAUIp ap majo9A un ysa *x no ‘g = (“x| ™)“A 
souvueA op jo g’x = ("xX | '4)“q ouuakour op uonnquy 
-sIp oun yUourepuadepur yuaatns ‘7 > y¥ mod ‘4 sanoy 
-egje sayqeuea say ‘7x yueYoes ‘aonb suosoddns ‘anbyeid 
3}xo}U0D UN sUep dLIONI} ke] JONsNyI xno op uygy 


IBIUT] UOISSI1dI1 Op sJdulex9 Us *p 


‘TUOLIZJUO ap 19 PBA eP SaqJ99 onb s}eyNsoI sMay]Iour 
ap 39 NOOS-oeY Ip spoujsu ev] e& sofqereduIOS sjyeI[NsaI sap 
auuop deqs}0og 3s3} op spoyjou anou onb ‘uonerinuns med 
opnje oun.p uaXoul ne ‘¢ UONdas BT & SUONUOW snou ‘anno 
uq ‘osdeuuC][HUeYIE .p ued np sonbysusjoeres sap a}dwi09 
sed juouuo ou tnb sanbissejo sjaroisoid op usAour ne 
JUDUIDTIORF Sagtnoyed aye jUsAned sonbysyeys soo ‘soguuOp 
-OINUI Op ISIYOI ds] SUep STUMO}J 939 UO deys}ooq sprod soy 
anb stoy oun ‘anb yey ne quon ("gH ¢ 4M ‘S) 7 dens}ooq 
anbysnejs ev] 39 (¢7) suep (9 5 °m ‘s)Z (oatd vou yuOU 
-g]JaMMUAA9) onbystye}s dus ey op uUONesSTTNN,| op adeyueAe 
jediouud a] ‘soOMjDeNMp sus] sUIZIxnep ke] op aidul0o 
J1ua} sed ou op stojred juatauod [I (1661) UOSTIAA 1° TRH 
QUBT[NOS JUO | SUIWIOZ ‘}S9} Np spNjyoexa,p NedATU 9] oIINpar 
yweLmod vya9 ‘stojajnoy, ‘aouesstind es juauIomessaoou sed 
}INP9l SU 39 Opl[eA UOU d9UUOTUOUISNS }s9} sp smMpgo0ld Fy] 
sed puas ou 30AId onbysyeys oun Jostyyn sed ou ‘sutourueaNy 
‘JHeULOJUI =jso osevUUOT[HURYOS | IS jyUoWIOJOT;NOTed 
joard un yuowonbnojduiAse juourayes9 elas (9 {Mm ‘S) 7 
aonb sed jyuowosnomoyyeul yueIes ou ead ‘densj}ooq 
}So} op amMpeooid smMoT]IOU sUN,.p UONUS}GO,] JUSUWOUTeYIIN 
ayploey JoArd un yUowonbyojdurkse ysa (3 { Q) 7 aonb yey a] 
‘oejses JUSUIZTeSA 1S9 ([66]) WOSTIM 12 [[eH op soudaMIp 
dus] SWgIxnep ev sioje Yoatd un yso (9 5 °M ‘S)7 IS 

(1661) UOSTLM 32 [RH Suep sguUOp soLOaIIp oust] 
aged ev] OUOP WeII[OIA 39 a]]NU ssayjodAy,] snos uon 
-NQUSIP &] JUSUA[qeUSAUOD sed jeID}d[JOI OU dIOTUIAp 91199 
ap uonesyun,T “(9 57M ‘s)7 wou yo ("GH SM ‘S) 7 
}so des}00q ne asturnos jsa nb anbysyeys ey anb suocjON 


(% ¢ ‘a]dwioxa Jed) v uONROTJIUBIS op jInas ne 
DINILIDJUT ISO NIRA k] IS a9}Afa1 so a][NU asamodAY J 


600z unl ‘ajanbua,p senbiuyoe | 


q 
{(9 5 om ‘s)a< ("GH SM 'S) 34 
uojas (d mayeA) dAJAasqo UOTeOTFIUSIS 
op [Ines 9g] Jaynoyeo ‘aynu asayjodAy,] ap jofor ne 
nal] suuop (9 { “A ‘s) 7 ap Moyea opueIs oun nbsind (m1 
S@ °° ‘T= q mod “(gH ¢ ,'M ‘s) 7 JaInoTRO (1 
‘go T=4 
9 S35 y¥ mod ‘,’m ‘densjooq spiod soy muajgo (1 
: JURAINS deys}00q $9} 
ap ampgdod ev] a19d8Ns eja_q ‘JowNrxo1dde suojnoa snou anb 
Q]]99 sIIp-e-}s9,9 ‘ay[nu ssempodAy,.| snos (9 {°M ‘s) J op 
d ja uw snos uONNqUSIp ey e onbyuapl swigUI-a][9 3sa Inb 
‘(dH {°M ‘s) 4s ap d 30 mw snos uoynaquysip ey] aonb ourgui 
e] jUouonbyojyduiAse yso ("GH S'M ‘s)7 op dens}ooq 
uoNGLASIp ek] ‘Z 39 | sjeynsar So] JUesT]yN usa ‘ouoq “yV 
mod d 3a wu snos jUd3I9AU0D }s9_Y ‘¢ 19 7 SasamjodAy sop 
miped y ‘oxouue us agjuasaid | jeyNsar Np d[99 & a]qQuiquias 
yo} 180 Q]]9 Jeo ‘7 yeINsI np sanaid e] suoWaWIO snoN 


‘9Haq]] ap 9189p 
un juepassod sayuepuadapul oues-1yy samoyegye soyqeuea 
sop nedsanou op jos Poy So] 30 GHZH),[CM ‘<) wy] 
= V op saidoid smoaea soy yuos ‘6 “ = b mod 
ov no Mey SZ <——— (“gH {im ‘s) ¢ 5 ZyDyNspy 

: yeqNSaI 

oUIdTXNSp UN suoUd}gO snoU ‘p ja ¢ SasaujodAY sa] snos 
“QyoINyOa 9p xneaey sInjny op yUoIspI0d 
Jonbg] Ins sutewop un 3sa ([°¢) Ip SOULI9} xNap So] JUd}OTJOI 
inb deys}oog sapoyjow ap uonelogeis.J (7661 10912 
ovy ‘ajduroxo red) sajqrey yuos gidap Jotwoid ap aseuuo][y 
-ueyoe p suonoey sa] onb asayjodAy,] snos snuajqo yUsANosS 
yuos deys}ooq sptod sa] jayja u_ “9[qIe} Zasse JUSUOATOOTZO 
jso SSLUUOT[NULYII .p sJeqo|s UOHoey ev] ‘sodeusul-saygnbua 
sosnoiquiou op sed aj sueq ‘ssenbydde ang jusanod ueyd 
3] Ins ssapuoj dens}oog sapoyjsur say aonb 3a saudoidde jso 
O"g) “A “a = Og) “A oneuntxordde | aonb ayos ap ‘(€007 
sy1aqoy re) ropurg NOA ‘ajduiaxa Jed) s[qeasijsau Issne 
In] JUSIADP ([°¢) Sp SIQUIDUT SWUATXNOp Np sULI9} sWeTxnop 
J] ‘sjqvestjsou jso AV /u aseuUOTTHULYIE.p s[eqo[s uon 
-oe] PT IS ‘sIOJaynoy ‘yeIauas ua ¢ asoyjodAy,] sed yuoJstes 
aU Saja ‘ouog ‘([°¢) op sIquIsU SWETXNap Np sUI9} Joruto1d 
3] onb juajsidap ou sajja.nb ayos op ‘oseuUOT[NUeYDE,p 
ueyd ne onp syyiqeuea ey onb yuaja]Jo1 ou prepur)s urd 9] 
Ins ssapuoj deys}00q sapoyjsul say “WaWIasnamMoyeyy “odeU 
-uo][HueYyse,p uejd ne.nb Isule s[apowl ne onp 9syI[IqeueA 

PR] Jojayjol nop deys}ooq uonnquysip ey] Ip juowenny 

(re) "9S a“A + (9) SA“ = ("A 

Inod d 3a W Snos 3jUdB1BAUOD }10S 


SM 
(° g)*A densjoog soueuea ey] onb Jastxa e yneatnbe asou} 
-odAy ayed ‘“ouIeJUSUIWIOD UN ayISsad9U ¢ aseuodAY.’] 


ongo[eye9 ne X-100-Z1 oN ‘epeueD anbysyeig 


‘(p'p'9 UONDAs Pe] Jat~noned ua 
: 9 amideyd “SG6T) NL 30 OBYS Ja}NsUOD ke dTAUT 4S9 IN9}O9q 9] 
‘saqiejap snd syuowoustesual sap nog ‘(anbyst8o] uorssers 
-91 op s[Epowl ‘oaMvoUT] UOISsaIs91 ap sjepour ‘atduioxa 
qed) sa[gpour 19 (766 JoNIg ep aseyUAsS op spore] 
SUEP Sa}LID9P saT]oo ‘ajdwioxe sed) deys}oog sapoyjsul sap 
uednid ve] dae sayreysnes a9 yaIeIAap jo 7 19 | SasomodAy 
sop dens}ooq sondoyeur sa] jos p 30 ¢ sasatjodAy saT 


"Z Inod d 3a w snos yUasIBAUOD ysa YX : ¢ asayjodAp] 


‘(Mm ‘s) wu mod deys}ooq 
UOTINQLYSIP Ud JUISIOAUOD 3Sd (im ‘s)yu : pasayjoday 


iad uf ap enbnoydursse 
dens}00g S0URLIRAOD-J0URUIRA op dOLyeUl eB] SO YX 30 
deys}oog UONNQMSIP Ud dUdSIOAUOD Fk] DUBISOp —— 
no (Z‘ON<—— (“d- “Pup — sc asampoddy 


: deqs}ooq spiod sap uononysuod ve] & SSANLIAI Sore} 
-uougjddns sasoyjodAy sioy aitnpoyut red suodsuswm0s 
snou ‘dejs}o0g 3s9} ap amp9ooid anou aIls9p op JUeAY 

“QJUSPIAN dJOTURW OUN,.p TUTZEp 
isa “*"g densjoog 9] red mayeumse ,g 9] « , » s]OqQuAAS 9] 
qed aousisap 4so 39 dens}oog uoNngmsip sgjadde jsa 19-a]]99 
:-M onb uonNqsip swig Be] UO Jo JUoUTepuadapuT soIy 
JUOS SM9}O9A g S9D JUOUIZID ,¥ Ino] suep ,’" dens}ooq 
sptod sa] jueuaju0D uv UOISUSUIP ap smMa}d0A g Sol ‘g 
SAPS GE PITLOM * gM Jed Issne suousisop snoN ‘sauuoT[O 
-ueyo9 sytun onbeyo mod “m densjooq uorsioa es red “m 
asepuos ap sprod 9] yuesejduisr ua “dg onb asarueul ouIgU 
RF] ap jyuongo,s **g dens}ooq 9] sed moyeumsa,J "yUdWI|9 
»Y wos suep ™m amoyegye deys}ooq spiod a] yUuatU0S Inb 
U UOISUSUNIP SP IM9}O9A 9] “M HOS 39 (766) T10919 OLY 
ap a]99 anb aja} dens}oog spoyjsu oun juRsITyN Us nus}qo 
‘y gqun,| inod amoyegye deys}ooq spiod un ™™ jlog 


agsodo.id dv.ys}00q spoyjoui ey “¢ 


‘assney ane yWeunod 
gynu osayjodAy,] onb uoneorput oun yso (Gx *M ‘S) J 
ap UONNQMsIpP eT op 9AI]9 a[WUSO UN ke JUOWOANeIedUIOD 
(2 <°m ‘s)7 op oantsod mofeA opueis oun,p uoNeAras 
-qO,] “uenbgsuos reg ‘JuIsod smofno} 3sa UID} SUIIISION 
3] ‘aAntsod aruyap ysa (*M ‘s) yw onbsind ‘anno uq ‘soay 
-LI9A JUdIOS Z 19 | sasomodAy So] aonb uoIpuos e& ‘USWIOA 
-yoadsar (u)“9 0 (uj) “0 (1)“Q. 24pi0,p ywos sau} 
SUIZISION 39 sWeIxnep “Jamuaid say onbsind ‘oyuowsne 
uo]WUeYIE .p ale} ej onb amsoul e sayne soy suUTUIOp 
(9°Z) Op BqUISWI SWIIXNeEp Np ULI} SUIZISION 9] “assnej 
yso ayfnu asamjoddy., 1s “(gH iM ‘S)g= (0 5 °m SS) 
SUOAR SnouU 32 jWUassieredsip (9°Z) op sIqUIOUI sWoTxnep 
Np soulia} sIatwiap xnap sat ‘ayjnu sseumoddAy,, snog 


ajanbus,p seguuop sap ins seseujodky,p $}s9} s9j Jnod anbiyesd desjs}oog apoujew uN : 1990g je JUOWNe|g Ze 


angoejZ9 ne X-[00-Z1 oN “epeueD onbysneig 


(9 — dH) {CM ‘s)y} (9 - dH) + 


(9-gH),_{C‘s)V}, (dH — "doz + 


(97) 


cS 


(dH {°M ‘S)z = (0 5°m SS) 
311199 ,S 
ynod (9 {ma ‘s)y ‘(€Z) Suep ‘onb JoUsT[NOSs ap yUsIAUOD 
I Sa} op amMpegdoid aNou Ins s[IejJep Sop ITWINO} op jURAY 
‘JURAINS U0T]IIS BP] B SUOALIO9p Snou 
anb 30 ‘deysiooq spiod sap juesi{yn ua anu ssoujodAy | 
snos (9 {*M ‘s) J ap UOLNQUsIp eB] SuoWTXoIdde snou ‘eI39 
ap nal] ny ‘(¢°Z) op SIQUIOUI SUIOTXNOp Np SoUIT}s9 s}UOWIOUI 
slotmoid xnop say 99Ae aploouo0d Inb ai[ayo9,] e STU 
9e9-Tyy Np uoyNquysip oun Jed (¢°Z) onb 9][9} UoNNquysIp 
oun sutxoidde yuo si] 189} op ainpgdoid Ina] sIMysuOD Mod 
ajqvreduioo jey[NseI uN asin WO ([Y6]) HOSS 10 ORY 


1=5 
cS 


BAS NOR Gare tO GA SY 
(6) 


(¢°2) 

ouop suoAe snou “(3 = GH UIp-e-1S9,9) IOTFLIDA 
}So aT[nu ssoyjodAy,| puend ‘oxouue ua sgjuasaid jsa ‘Z 390 
[ sosojodAy soy ins atndde,s mb ‘| 3ey[nso1 np sAnaid eT 


‘QHOqI] op adap un yuepessod oud 
“TY sarfoyegye safqeuea sop quos °7y 19 (HXH)(,-V) 
= VY op saidoid smoea so] yuos ‘GO “ = b mod 
°y no “es’y Sh <—a— (SH i'M ‘Sy: [ woynsay 
: JURAINS }e}[NSOI 9] JastjyN eyad nod suoanod snow ‘ay;Nu 
asoujodAy,| snos (9 { “a ‘S) 7 ap UoNNQUsIp e] Jowxodde 
SUOADP SnoU ‘oplyeA }S9} op snpgdoid oun ImU3}qQo Mog 
‘aseuUoT[HUeYsE .p urd 
3] 19 aJapour aq snos '“"L7 no ;X% wuonnquysip e] sed juowas 
-nomMoyjeul jins ou (p'Z) aonb nuuod udIq }jSe JI ‘9IJLIDA 3S9 
gnu asoyjodAy,| 1S ‘prepueys somnpgooid say suep [LHOIAM 
UOTONASUI,| JURSTTN Ud jUdI}QO.S (p'Z) 3S} op onbysne}s 
FI “SVS swaiscs 9] JosI[yN,p Wstoyo uo] Ig “7 esetpoddy,| 
OLJSHLS op Sd ‘oLIE|NZaI op SUOTIPUOD sone .p e onsnfucs 
‘Moov 99 Op sJOI aJ ‘wWeredsip ([ — 4)/([ — NA) ANa}0R] 9] 
‘sqysATeue So] JUSWAT[oMIGQeYy yUOoFJ onb 99 ‘u = i keond 
UWI SP JOS J S[I,g ‘sguUO[IY999I1 JUOS sprod say yUOp 
uodsey e] op sed puedsp ou mb ‘(av/""g) {1 — 4/(I — W)} 
sak Abe) V 4sa ajuaoel-snos agispuod ajay, & 
ast ap soLjeur ef ‘(p'Z) sue “(IT — W)/,C"g — 4) mE 
=e Ora i Sine Diy Ra mg MZ = NNO 


NTO) {CL - /-— ADF 
1° Gs a) 
JULIO} B] SNOS s9I9pUOd js9} op onbysye}s ayjou WUEp 
suoAnod snou ‘suuonjuouwsns oajdwioxa., wWeAInsmog 
‘ajgnbua p saguuop ap sinayesi[yn say] nod spouwos 
snjd }sa inb a0 ‘asepuos ap spiod uos sed uonearasqo 
onbeys op uoneispuod re] juayouuied sfi.nb uonipuoo 


(v2) 


(9 5 °m ‘s) 3 


Le 


e ‘sanbissejo sparoisoid sop juest[N Ud JUSUII}DAIIP sgTNo[eo 
ang sioje ynod (9 {*m ‘s) J ayuURINSoI so} op onbysnes 
eT nb ysa (S) W = (°T ‘S) y aonb aqjar ("mM ‘s) W aq]eY99,] 
® osIW op sOLYeUI ouN.p uonestyn | op sodejurar sop un | 
‘(¢°Z) uonenbs,] wns nb sydeisered ne suouuoNUsU 9] snou 
suUIWIOD, ‘aseUUOT[HUeYDE.p ued ne jo s[apow ne yodde1 
red sonjead “Gx op soueLeA ey] ("gH “A ap d ja 
UL SNos yUaBIBAU0D Inayeurmsa un 3sa nb ‘(*" gy) cae qed 
(¢'z) suep (°M ‘s) Y yuedeydual ua jUd1}G0,] uO ‘oyUeIN0D 
PIeM oP onbusne\s B] Jsd xIOYD anne UL) ([86I 09S 
yo ovy ‘ajduraxe sed) ajdums amoyesye oseuUOT]HURYE snos 
“dH op ueyd ne yoddei red soueea ev] op Inayewmso un 
red a9oe|duiar ane yreumod aya ‘ajduraxa eg ‘sayqissod juos 
(€°Z) suep (“M ‘s) y aT[OYO9,] B STU op 2oLNLUT eT Op xIOYS 
sonne.q “y mod ld SNOS 91U9B19AU09 JUdUIATTOMIIQeY BIOS 
(Q) WN 4aqjo uq ‘(Q)V mod d snos a}ua3I1dAU0d9 }10S 
(7M ‘S)V anb sed a3srxa.u 7 asomjoddy,| anb suojon 


‘y ‘onbunjourss jo sATyISod arutyfap ‘axIf d]]9499 | 
® Osi op ooLyeul aun mod d ja w snos qUasIBAUOD 39 
sanisod armyap ‘onbrunoutss so (*M ‘S) Wu : 7 asayjoddy] 
‘aystiqeqoid oSeuuo]HUEYE,p urd 
3] ousIsap « d » uOoNR}OU P| onb sIpuR} ‘o]Epoul 9] oUSISap 
« UW >» UOTROU eT ‘aseUUOTTHULYOS,p ued J] 39 dTQ=pour 
ag] snos ‘"g | ul op onbnojduiAse s0ueleAod-a0uRUeA 
op soLeu eT Iso ZX 39 ‘AdeUUO][NHUeYDI,.p ud 9] 39 S[apoUT 
3] SNOS UONNQMsIP Ua sdUdSIDAUOD LI DUSISsp <—z-— 
no (X‘O)N<—ae—_ (I - “d) uf —: | asemsodayy 
: SODIJLIDA JUSTOS Sa]URAINS sasoujodAy 
xnop soy onb alguew op sjinqsuod juos asepuos op 
sptod sa] 39 aseuUCT|HUeYD9.p utd ay] aonb suosoddns snoyy 
(7661 Tepures 30 aT]IAoq ‘ajduiaxa sed) onuuod ouia}xa 
uoneuuojul | ap ajduwioo mua} mod ages ane sioje ynod 
nb ‘uondaT9s ap ai]Iqeqold es ap ASIOAUT| B [edd 4SO ‘s SB ¥ 
oyun oun mod ‘%m™ asepuos ap spiod 9] “uowoneUIpIO 
‘asepuos ap sprod uos red uoneArasqgo onbeys ap 
uolelopuod ev] ayouued Jaap 90 anb uonIpuoo e ‘faro1so01d 
SUIQUI 9] JURSITN Ud dgTNoTeo ang Issne nad (9 ‘ *M ‘s)7 
onbysnejs et ‘ajgnbua.p sasuuop soap ioyen mod 
addojaaap 939 yUoWaessac9u sed vu Inb onbissejo jaro1s01d 
un,p JURAIIS as Ud sgTNoTed ang ynoed (9 ‘s)7 onbysye}s 
epson (a [s) gi ss) PO SUOARE SHOE Jou tases a 
9p gigpuod onsoyeue un issne yso (9 {mM ‘s) 7 onbysneis 
eR] Yuonbgsuos Jeg ‘UN smMajeA op uUOTTWURYDS Ima}d9A uN 
yso “[ no “(s) VW = (‘I ‘s) y oUOp suoAR snoN ‘(S) W 99AR 
nal] &.U UOneIopuod aumone nb sioje ‘%" asepuos ap sprod 
uos red sgispuod jsa y a9uUOT]WUeYdE aun onbeyo onb 
suas 90 ud (S) YW op aiapuod ondojeue un jso (*M ‘S) Y¥ 
12 g op siapuod mayeumsa un yso ““g ‘mu sed audisop 
‘JUOW9[9 ,¥ UOS SURP y¥ s9UUOT[HUeYDE yun] op osepuos 
ap spiod a] yUNUOD u UOISUSUTIP op “M IMI}O9A JT 


(£2) (@— “PHD {CM ‘S)v},00 — “GHD = (0 5° ‘s) 7 


6002 uln! ‘a}gnbus,p sanbiuyoo] 


SULIO} P| Sp dJ9pUOd js9} op onbysyR}s oun JasTyN.p 
Suonbey snjd yuousjqeyiqnput yso mb 99 ‘otidoidde 
snjd ong weunod | ‘ojapour ne yodde sed jneunojym 
so adeuUOT[HULYE,| Is Juepuadag ([ — u)/,("g — 74) "KX 
= "9 » wi '"Z="g no “9/(9- “glu =('s)2 
B ywesousw vlad ‘ouUOMUSUISns a]duWIoxe | sueq ‘(9‘/)/ 
aonb joynd  (9‘s)3 onbysnejs ey JosyNN.p s[qeiajoid ane 
yweLmod jI ‘ssguuoy]yUeyos uOU s9yIUN sa] od sagArasqo 
sed jUOS 9U X SajyUepusdapul saTqeLIeA sa] UoWIdTToMUSAg 
40 4 oayuepuodsp osjqeuea ey onbsing ‘(s)d 9uuop 
aysyiqeqoid aseuuo]]QUeyos,p ued un e jJUdUTSULIOJUOD 
Q suy uonendod ev] op 9m 3sa uw ale} op s oanojyegye 
uo]]WUeYoS UN ‘sed da] JUSWIANeUIPIO 3Sd eIIO UIWIOD 

‘oT[NU ssomodAy | 
Snos Ny 10 2% uonnquysip ke] yIns onbysyes aye0 
anb juaulaTjamarqey asoddns ug “Vv/"9 = (Q) PV anb suosod 
snou Is ({°Z) anb sunoj owigu Pe] ap so (7'Z) onbysyeys 
BLN ds 1) =,%9..39, Wieder = Og. no 


| cid! Linrareic, « 
(72) e-*g) ~ (9° A)s 


3Sa awW9]qQoOId 39 Mod 3yueM09 3s9} ap onbystye}s su ‘uN 
smngjeA ap uoyeindod ap mayooa unyso “fT no ‘79y = “¥X 39 
fa =. | =O duloxo joo sucq 9 =.g ° “Ty ornu 
asomodAy J 19383} suoyeyNnos snou onb 39 ‘g DOURLIeA ap 39 
g ouusAoul op ssanqiysip yuowonbyuap! 39 sayuepuadapul 
SomoyegTe Sa[qeueA sap yuos ‘7 > ¥ UOHeINdod zy] ap soyrun 
Sa] Sajno} mod ‘*4 say aonb suosoddns ‘ajdwiexa.p ann V 
‘sayIsod atulap 3a onbunausss 3s ( ADV 
JuoWaTaMIgeH “a]]oYo9 | & SSTUI ap sOLNeUT 9UN 3S9 ()V 
32 JJ2pouw a] snos ¥g ap JUasIZAUOD MayeUIT]Sa UN 3S “1g no 


‘(a -— “gH) {CAV}, - dH) = @ £1)? 


: yUeAINS onbyeIpenb suuoj e] yuo nb sonby 
-S]R}S SI] SUOIDPISUOD SNON] ‘nuUOOUT sHoWRIed uNdNep 
puodsp su nb onbnojyduiAse uoynqiysip oun jyIns a]9 “Ip 
quswogne ‘yoAId un yuswonbyojdurcse yso (3 { 9) J ‘WUoW 
-g[eap] ‘oysAyeue | Ted sogiyioads sayuejsuod op CG UOIsUdUI 
-IP OP M9JO9A UN 4Sd9 I 39 19}S9} B asaJOdAY, | WUIZap mod 29s 
-I]qn jso 4x OG suoisuounp op W compeul ey 9 + GH: 'H 
saneulayye asamodsy.] anuoo 3= AGH: °H 9ZIdH[nW 
omesut] aseyjodAy,| 193893 mod (3 {)s 389} op onbysyeys 
oun yelasyyn uo ‘fQ aa{duioo uoneindod er] JaArosqo 
yteanod uo] Is ‘onb juauayesa suosoddns snoN ‘g op uoU 
steul ‘g op jofns ne sasamjodAy sap J0}S9} suO}eyNOs SNON 
‘Q]apoul np sameyuous;ddns snuuoour sanouresed op onus) 
-od ma}99A UN jsd @ 39 J[QPOW Np snuUOOU soyoWeIed 
Sop 4 UOISUSUIP op ma}d0A uN so g ‘xX soayuepuodapul 
SojqeueA ap majo9A uN.p uoyeindod ev suep smMoeA soy 
yUstUOS Inb sausI] AV e VoLYeUT oUN Ysa “YK ‘4 ayUepuadap 
g[qeueA oun.p uoneindod ev, suep smojeA soy jUOTJUOD 


(12) 


angojeyed ne X-100-ZI ,.N ‘epeued onbnsneis 


N uorsuourp op 7A moajooa oy (9 ‘ff ?K| 2A) y 
g]JouuoNIpuod §=uoNnqmsIp ke] woop mb ‘ayscjeue,] 
red g1yloods ‘ajapoul un ke jUdWIOULIOJUOD 99910 9219 Fb 
N 2[te1 9p Q ouwy uoneyndod oun.nb suosoddns sno 


saTeUNUye1g °7 


‘UOISSNOSIP duN 39 SWINSaI Jorg un Jed 
SUON[IUOD snou “UOTdas dINTWAp ey] eB ‘UIJUq ‘suosodoid snou 
anb dens}oog spoujow e] e& sucIeduIOd sa] 19 SapoyJeUT S90 
suoseduiod snou ‘uoneinuns red apme oun,.p usfoul ne ‘9 
UONIIS B] YW “UIVOUT] UOISSoIS9I Op d[apou uN,p sanourered 
sop IN9}90A un_.p jofns ne sasoyjodAy,p $}sa} ap seo a] suep 
‘asueYdaI OP SOpousu sop jos Inb ‘TuoMayUOg ap 19 PleA\ 
2p “(1861 NOIg 19 ORY) NOdS-oVY ap sapoujsUT sa] JUOW 
-OA9LIQ SUOALID9P snou “¢ UOT}DaS BF] WY ‘aUO9} ke] JaNSNyyI 
Inod oalesul] uoIssoiso1 op ajduroxa un suouuop snou 
‘p UONDAS BF] Y ‘aJonbua,p saguuop sap med e sasayjodAy,p 
$}S3} S39] mod suosodoid snou onb dens}o0q spoujow 
eR] suoynsnf 39 SUOALIDap snou ‘¢ UOT}DES FB] YW ‘“OWIIIqG 
-old 93] 39 UONR}OU ek] sUOJUassId snoU ‘Z uonoas RV 
“saxo[ duro sajgnbua .p 
saguUOp sop yUoUID}IeN a] Mod sasteroads sja1o1so] ap sed 
jUSsOdsIp dU S]I,s SWI9UI UoWaTIOR} san Jonbrdde | yuaanod 
saysAyTeue say ‘dens}ooqg spiod ap sajquiasua say ndol MOA 
soide ‘aonb sd ajgnbua.p saouuop sop med e sasoyjodAy,p 
S$}S3} Op sayuRstxd sopoyjour xne pyodder red deys}ooq 
poyj}sw aNOU op sasejURAe Sop UN, ‘dens}ooq spiod sap 
SUOSITI]N snoU ‘aT9pour UN Ins sagpuo} sagigpuod sanbysye}s 
soo ap anu asomodAy,] snos uonnqmsip ey] Jowrxoidde 
Inog ‘sonbtsseyd sjarsisoid ap spre. e JWoWIaTIOKy yUaTNITed 
as sonbysnejs soo uowajomiqeyH ‘asepuos ap spiod 
Sd] SoIS9jUI yUOS sayjonbsa_ suep s[apoul uN Ins sogpuoy 
sso} op sonbysnejs sop ims omndde.s spomouw onon 
‘JHeuLOJUT sed ysa,u aseuUOT[NUeYIE | ‘UoUTs ‘ UoNe[ndod 
R] Op a]quiasua,] mod yuan nb myjso ap aayIp 9uUONdeI9s 
uo]HUeYII,| Inod sIyIDA sJapoul os] puenb JneuUoyur so 
aseuuoTHUeYS | Guepred JUSWIDT[OWLIOJUT “WOU NO JHeWOJUI 
aye jnod uoyHueYyos,] JouUOTda]9s Inod gsi[HN asI[Iqeqoid 
sseuuo]]NueYoS p urd a] ‘army uonejndod oun.p om 919 
® UoT|HUeYOS,| puenb sjapou un.p snuuosul sanourered op 
3na}99A un.p yafns ne sasapjodAy,p s}s3} saq Inod densjooq 
apoyjeur oun Josodoid ap suop 3s9 aore yuosoid np jnoof 
-qO,J ‘sasaujodAy ,p s}sa} soy Inod sgIpm9 939 OAR a100U 
sed ajquios ou deysj}oog spoyjou ve] ‘sopoyjou soo sup 
JUCUDAIDJUT SONUUOSUT SddURLIRAOD-SOOURUBA Sd] JOUITISO 
mod a9-jnod jnes ‘stojaynoy (€OOT SeWIOYL, 19 OVY * 9661 
SHaqoY 12 YsUIS ‘seWIOY], * £66] WOy 19 preqneIyH * 1661 
preqnely 39 WO 066] PleqneIH 19 WIOY * 86] COPY 39 
SBUIOULL £ S861 ABA S 186] NOS 19 ORY s[dwoxe sed) saguue 
SdIIWIDP OF SI] sIpmMe usdIq 99 e soxajdwi0s s}gnbus,p 
saguuop sap Impyed e sasamjodAy,p s}so} sap owigqoid oT 


a}enbus,p sasuUOp sap ins sasaujodAy,p s}sa} Sa] Nod enbiyesd desjs}oog spoujew suf : 990g }e JuOWNe|g O€ 


:J9wINCOD «“OLOVINA ‘epeueg ‘(ouRUO) eMENO ‘23R9,11 


‘9180D-"H-Y 8] qnoeuwru1 


"p0'08 URdIEISM)1IN90g ‘eIy}UAD 


‘Aguuny, gig ‘sasudanua sap saidne ajgnbus.p 


SOpoy}sul sop UOISIAIG ‘epeuRD anbysHeg ‘900g eIyjUD ‘ ed'03'ULD}RIs@)UOUINKIg slooURIJ-URIL : PLUNOD “9LO VIM ‘epeurD ‘(ouRjUO) eMeENO 
‘93R19 ,9| ‘SVOD-"H-'Y 21qnowun ‘Aouuny sig ‘onbysyejs us UONPAOUUT,| Op 19 BYOIOYIOI B] Sp UOISIAIG ‘epeuRD onbysHeyg “UOUNeag sIoSueIj-uesL “| 


JHeULOsUr 
sed 3sa.U a8euuo][NUeYS,| puenb opryea jso ainpgo01d ma] 
‘soUBITNOS ao,p JoWesa JUDILIOUI ‘onbysIdo] UOTssoIs9I 
ap japour un,p juowajsnfe, 19}s9} Imod agnbus.p 
saguuop sap e onbissejo onbunjsurered densj}ooq spoujow 
e] anbrdde yuo inb “(661) SuNIppI] 19 Wioy ‘preqneiy 
op xneaey sa] ‘anbissejo onbysnejs ua dens}oog spoujow 
PR] Op npual a}dwios yuaT[I0xe un ‘(¢ GG) TueITYSQI] 19 UO 
suep 30 dens}ooq sasoujodAy,p sso} sop uoIssnosIp oun 
(1661) UOSTIM 19 [[eH Suep eIDANON M9199] YT ‘onbisse[s 
onbysneys B] ap a}XxXo]U09 J] sUeP SIPME 919 ye sWIETqQoId 
3] onbionb ‘sasepuos sap sed oy suep densj}ooq spoujoul 
eB] JUeST]N sasomjodAy,p s}so} soy Ins uonesqnd sunone 
Ja\STXO gIQUIDS OU ]I ‘sUOTYDesS snou onb juURINY (7661 
T109 19 OFY = 7661 JONIS = 8861 NAA 19 OBY TAOY * 8861 
NA 39 oVy ‘a]duroxa sed) omy uonefndod op snowed 
un mod deys}o0g soueIJUOS ap SaTTBAIOIUI Sop UONPUTWID}9p 
B] ap auajqoid a] sIpme Juowyesa yUO sInayne suTeyIdD 
‘(Sapoyjaul sop onaar oun mod ‘g onideyo ‘sE6t NE 10 
ORYS NO 766 JOIS MOA) ory UoNeTNdod oun, p dsstwI9I sues 
911} JS9 UOTTHUeYII,| puenb soueLIeA ev] JouNsa mod soud 
-oidde deys}ooq spoyjow sun IMIANODEp ke yUoUTaTedIOUUd 
QSIA JUO xneAe So] ‘OYdIOYOOI ek] Op ouUIeWIOp a] sue 
‘(p66 AYSporg 19 soysNH * G66] NYT 19 JowuRY “024A * €00T 


asesa] 39 Joyoney Jojduey] ‘ojdwioxo red) soueueA ey] op 
uOneUINsS p soua{qoid xne juowsTediouud sg,TUNT] 919 IIOAR 
gjquias onbyeid ua spool ey op UONesTTNN | ‘tor. nbsne 
‘soidop smoisnyd 
eB ayneNs oseuuo][nueyog,p ued un,p sed a] suep asemodky 
ana9 snos deys}ooq spiod op uonjonnsuod op spoujow 
aun .p ‘aieyo steul ‘ajyouTIons uoNduUdsap oun (766[) NA 
19 NAA ‘OBY SUEP BIDANO MI}d9] YT “sTWaI SdAe OSeUUOTIN 
-ueyos.p ued un sed juowajd0uI09 sutxoidde ang assind 
asa sues aseuUOCTTHUeYyoS.p urd un.nb mod soqrey 
juswesyjns yUOs sidop Jomuoid op osseuUOTTWUeYDE,p 
suonoey say aonb yuesoddns ua snuajqo JUsWIATTaMIQey JOS 
sptod saz ‘aguuop 3s9,U uryd 9] Ins UONeUOJUI anne suNoNe 
‘ qenbua.p sasuUOpOIONU ap IOTYOIy a] DOAe SaysATeURe xne 
STUIMO} JUOS deNns}ooq sprod ap sayquiasua smotsnyd ‘suones 
-1]dde sas op yednyd ey sueq ‘saysAyeue soy nod spoumu0s 
ysd ayja.nb ja (sassy] uou sanbysnejs ‘aseyes ‘asuoda1 
-uou e] mod spiod sop juauiajsnfe ajdwoxo sed) juow 
-aQNe sIPNoOsal sp IIOYJIp wWesas [I.nb suonemis smorsnyd 
JgyIeQ op youuiad ayjo.nb ang jUa;quias suoste1 soyediouud 
SY] “xneuoneu sonbysnejis sowstuesio soy Jed sagsiyesl 
asepuos red sajygnbua sop 3}x3}U09 a] suep JuoWUONbay 
snjd ua snjd op oasijyn jso dens}ooq spomow ey 


uoyHINpo.yuy = *T 


‘Q[9poul uN, p sayouesed { olle9Ul] UOISssaId91 
‘JHeuuojUI oseuUOT[WUeYID ‘ sasoujoddy.p 3sa} ‘ajgnbua.p saguuop ap asAjeue ‘ dens}o0oq splog 


> S9J9 SJOJ 


“OIIVQUI] UOISSAIZ9I Op s[9poul uN, p senouIesed ap 1n9}99A uN,p jafns 
ne sasoyjodAy sop o)89} uo puenb tuoLajuog ap 39 pre 9p SepoyjoWI Sap xnad anb sIna]]I9UI 39 WOIS-ORY Ip spoujou eB] op 
XNd0 & Safqeiedwiod s}e][NSg1 Sop sUUOP spoyjoUL ke] ‘a]quIasUd | SURp ‘onb aIpUd\Ud JUASSIL] UOTJE[NUIS op s}eI[NS9I SoU ‘aNO 
uq ‘soxoduios sajygnbus.p saguuOp sap juouo}Iey a] nod sastje1oads sjo1o1Zo] ap sed juasodsip ou sjI,s sWIgW UOUOTIORy 
soy Jonbidde | yuaanod saysAyeue soy ‘deys}ooq spiod ap sajquiasua sa] ndoi MoAe soide.nb yso ajgnbua,.p saguuop sop 
Jmed e sasoyjodAy.p so} op soyue\sIxd sapoujou xne poddes sed deysj}oog spoyjeul ayou ap saseyueAe sop un,’T ‘deys}o0q 
spiod sop jUeBsI[IN Us sJQ9pou UN INS sdgpuo} sagigpuOd sonbysije}s sao op aj[nu asayjodAy | snos UOINQU)sIp e] suowxoidde 
SNON ‘sonbisse[d sonbysyejs sjororsoid op yuRAIOS 9S Ud IO[NI[eO e SaTlOv} JUSWIDTJoOMIQeY yUOS sonbysyE}s sag ‘odepuOsS 
op spiod sa] soisajuUI yUOS sajjanbsa] suep sJapoul uN Ins so9puUoj 3s9} op sanbysye}s sap Ins ainddes spoyjy9uI ANON “UoU no 
jJHeULOJUI og jnod UOT[WURYD9,| JOUUONDaI9s Inod gsI]I]N as1]Iqeqoid aseuUO][HULYOS,.p ued a] ‘o1uy UoNe[ndod aun,p 919 
919 B UO][NHURYD,| puenb sJapow uN.p snuUOSUT sayoWered op Ind}09A uN,p jafns ne sasayjodAy,p s}so} sa] nod deys}o0q 
spoyjsui oun suosodoid snou ‘oporpe juasgid 3] sueq “soURLIeA eB] OP UOTJRUIT]SO,p soula[qold xne JusWajediouLd saytuNI] 919 
JOA® 3]qQuIas snbyeid us spoyjsU ek] Op UONRsI[NN,| Guosgid e .nbsne ‘ajgnbus,p saguUOPOIOIU ap JaIYOI o] OdAe saysATeuR 
xne stwinoj juos deys}ooq spiod ap sojquissua sinaisnid ‘suoneordde sop pednid e] sueq ‘xneuoneu sonbysye}s sourstued10 
So] Jed sagstjegi osepuos Jed sajgnbua sap 9}xa}U09 a] suep npuedai snjd ua snd ap jso deys}ooq spoujou P| & SINODII FT] 


guinsoy 


, 90g emQUdZD 30 JUOUNLIY sIosuLAy-UvOLS 


dj9nbus,p soguuOp sop aAnAed v 
sasayjodAy ,p $}s9} soy Anod anbyuead deijs}00q apoyo sup 


6c 


anbojeyeo ne X-1.00-Z1 .N ‘epeued anbysye}s 
Ov-6z “dd ‘| .N ‘GE IOA 
600z uin! ‘ayanbuea,p sanbiuyoe |) 


A i by lagen? or . petirmendone meat yny eSxrpet ees ayating 


ies Met 1) Lee 
a, ae, ? bo ee atiaiitc on j 
’ y mes ee jen tie ine hae: ny 
“ait a a erly 38 o FRR. S MOMVETT Hate Wa Y UNanes Ut oy sateul 
MRT em PA SIA ap YNpErs Muss 7] 
ie i a eo ob: VARA 
fat i 1 : Aa aco ant pedi tease OP TA dels lit hand ‘vations Me). GomT gmat 
ade Roy wh im ot 19 fa aa Suen" q  eepPupey ‘RA Ry ARAM See i 
; » of i, eee eee SANS Werks wa nahi Huns sera d 7 '0C dy 
{ Reewaen ong tion F cena) oy ayrsnaeni ef (etaorudtee w vy soeldy Hatta, {30 
so non Ko mee MGT ORE ra daly itp cipro ome sepia spe sor SF joH0 
ret is Gohanstng SARA ieee "s eot sar ky 9 May wd wedtes vores | viet 
y ery ayn ? Uwuy ani yi 
fn wom = 25, oA Va "apy 


Mikp snarls pap agentes: ev sp 

SPUD ak HaatORORYS Spo Lap sO IU; 

me ONL a 1 cba 

Be Sa ser 24 3038 LeRuSevAONl MUCH WIOMIADT! \y° eft jae" 
ad b 


iq 7 oF OG SG ASWApG Fiptetwe a oer 
7 Fak arr ‘psa fea 16005) PYM hy i] [its H 
Heme SF ahs 


ey tetien: Solin at.d, 2320 7331 : ‘ a sf 
af wn i ee Mk ge coaveers aaa oa se we HR rey 
isan ir (Sawey edi mines i fine vidas BO nt pn: Spy Dd ae BED, boty HR [RYO TRS 
PERL IR amnranen age NTT IL Rare i SEMA RAB hid 
Ri ts rice be SM =f Rafi jWaeaar) sel wettest a | 25 cape 


+ 


of on ote i. at: eee wot ag Ath ree F) “wicapigo cm) A oe ‘ = f° i 5 : 
- at Leta: at} Min) ty i ndiahr iasar! ‘ ead” yt aide ee we toe via leat \ ‘ 
ai Ieee er ye ies - tan hil Haru i sith Rig i aormmnnanys ep Signa? rise cree Faris EMRE 


ras Ft RAST (wa Qty) MO IMMAANS I Ag Koy Soe De 
fihin scan n = PEL ela Ye ao so Pee co eu anal a 
yap ar iets a eee oemerbnt |) eet oh gress 
ii seyiea Bh th wowetules i" r, 
by (n aorta, oapnertin> CRS Ay QUE) oa sy pared % guoe’ Bs Uap Jaenny: 
ia - Ce baspo) TBR: ipsa nt Veen ig ea a oe GATS SVG. 8] aPC yt YWow’ mae [ea x Gey t jou 
: Aare Demian | a hess" CN Ci) Eee Vat eaetes 
Paces ey aan Lt igarwr wpm : | ‘ 
4 AP eampetes | encore ice d mre SHR OR AG Sk BRT Leroy sap 


x Cade fis The TE. smyop, ee psa dane A ata, {payer aenracnh sy ow 

: eT ee cre ee a AIR: AR nee SERA WS rebaestit 

| s ed * —* T 
ren ur eoare wonton My OWT Ch hart ea Sidi RAGING p om ny roubnstt 


ants ab “(£) anos Shei oy peda sad STEM; oe feet vr igh, QP AAU WT) UH aint 
ART) Gut, sia > (ON? GAS sf at Be i it an Aah FERNS Syl 
c BAteag HA sth ee UD ey Ry" sae PEAR ioe Age ia 43 % 2 aia th. a Satan iy 
Pein r (aap cae vtec age ge kp Made ae im bes Mh i de, t at A AU 7 |} ‘ 1Aca) SADA Gy 
Pathan das TIS EY) wags $= © iyutatae “hab ae Oe Sy yun <) eM eNO 


fa AR AR a = sates Hist np “ied 2a ewes ‘9 Sepa 


ist A®, { OLR, ‘gh Sho. Oe 
id Heiig Hop ar 2909 (p) Ram Teme ¢ SRL oe amen are Cah a) a 
e265AL re: on hanes, a eee a ee. te. 
wu, cate i i Pa 7 pyc’ beeen ae aod 
: Che wwe 3 yaar y 
y ou ah rene theft 4 ah fey 4 
PRK ES fT ‘agin ios PEA 


rach ore voit 
LEPC ELARE EN | TURE WORT 


adenauan Hey” 

a gBaKe' A aa atte ote A onesaeles 

‘ P) wade Wows ie en | ay gh os , 
“yw is Ye ae 


on thee sores 


hind 


oie a 


andoyejeo Ne X-[00-ZI .N ‘epeued anbysneig 


‘OPL-IPL ‘6L ‘DYlMawolg ‘S[2poul UOISsoIdaI 
IeoUl] Ul UONoIJas s[qeuUeA Joye souangUT (7661) ‘qd ‘“sueyZ 


"698-858 © 96 ‘UolDIDOSSy 
Jooysyojg uvdidaup ayy fo jouinor ‘AaAing uonesounua 
ysod 0661 24) 0) UoNeoyddy :sAoAIns Ul sJo\sn[o [eNuonyul 


SUNYSIOMUMOG (1007) “UL “WP 39 "N “Jayuayos “WV “AYSAe|seZ 


‘8T-ELP 09 ‘TeoIsojopomay “gq Souag ‘Aja100g 
joaysyojg jodoy ays fo joUANOL *S191}JNO Zututeyuod SASAINS 9] dures 


Woy UoHeUINSse payesqieo-selg (3661) “A ‘MEyoUoY 19 “H'V “YSTOM 


“uy ‘suUOS 2p AdTIAA UYOS : YIOX MON 
UONIPS SUIQISIOL], ‘uoissassay AvauiT patjddp (SOOT) 'S “B1OqSIOM 


“OUT “SUOSg 2p A2TIAA 
uyor : YIOX MIN ‘Yovooiddp uoyoipadd p :aduasaf{uy pup suijduvy 
uoynjndog aiiut-s (0007) We ‘TWekoy 19 “H’V “ueugsog “Y ‘juer[eA 


‘OUT “SUOS 2p ADT UYOS : HOA MON “(UPS “TL 
9 YOH q ‘1ouurs fa ‘spq) ‘sdaaing xajduoy fo sisdjoupy 
suep 9 onideyd “q ueg 0} uoNonponuy (6861) WL ‘WtwsS 


TS I-Evl ‘ pl ‘sousuwis payddy fo jousnor 
‘Suljdues AdAIns Ul SUONRAIOSGgO JeQuONyU] (2861) AW'L ‘ytus 


‘OUT “SUOS 2p AdTIAA UYOS : YIOX MON “sdaaung xajduoy 
Jo sistjouy (6861) (Sp?) “WL “WMS 19 “qd “YOH “fO “uurys 


OU] ‘sug 2p AdTIA\ UYOL 
> IOX MON ‘(JouUN{S ‘_ 10 sioquieyD “Y spy) ‘vInq daaung fo 
sisdjouy suep 9 anideyD “g ued 0} UoNONpoNU (€00Z) “f'D ‘JoUULyS 


"E9L-8CL “UOHPIOOSSY [BONSIE}IS UBOLIOUTY ‘spoysayy 
yospasay Aaaing uo uoysag ays fo ssurpaa201g ‘soyeuttysa 
AQAINS JESUT[UOU UO BUTWILY JYBIIM JO J9JO sy “(€661) ‘fy “IaN0g 


‘O€7Z-SZZ ‘UONRIDOSsy [BONSHRIS ULOLIOWY ‘spoysayy 
yoapasay Aaaing uo uoijdag ays fo ssuipaaz04g *S\y3iom Burydures 
SUWI9YX9 WILY pue AJQUSpr 0} sonpaooid Jo Apmis Y (0661) ‘fF “Jan0d 


“197-6£T “S “YoAvasey [DIIpay ul Spoysapy joIuUSIIDIg “sisKjeue 
Bep ADAINS JO} syYSIoM Surjdwes Jo asn sy] (966) “q ‘UUeULIATZa} g 


‘876-€76 (98 ‘DYljawmolg 
‘seq [RUOHIPUOD :Suljdwes Adams ul dysouseIp sousnU] (6661) 
‘f ‘opreysig-zoun,y jo “y ‘sokay-zounpw, “Tf ‘oT]Oqay-ousI0jy 


Le 


dse-yxipuoddy/3 ¢6¢€-poO VINS/sand]ye/suonesrqnd/Aos esyures 
‘yyeoueyuow//:dyy ne sjqmuodsiq ‘uoyeysIUUIpy sddIAIog 
(peoH eal pue osnqy soursqng - WSN CW AIllAjoy 
‘8E6E-POWNS “ON UonROGNd SHHA “Z007 ‘seg pou, 
‘yeoH [esl (7007) FW “UosiopusH 30 “MY ‘Ployosiopuey| 


“LECE-OEEE ‘UONLIOOSSY [BONsHeIS UROLIOUTY ‘spoyjayy YosDasay 
daaing uo uoysas ay) fo ssuipaadoig ‘syysiom Burdures YIM 
UOISSIIBOI Jeoul] ul sisAjeue soudnyuy (9007) “Y Jueyed 1 “¢ ‘TT 


‘puryAreyy JO AVISIZAtUy ‘dat;qnd 
-UOU }e10}NOP ap asa] ‘suUOyWasasq<: joOIjuanjfuy fo uoywotfiyjuap] 
oq Aaaing xajdwoy sof soysousviq uoissaisay (LOOT) ‘f ‘VT 


OU] “SUS 2p ADTIAA UYOS : HOA 
MON ‘sdaaing ysoafy fo sistjpuy (6661) Td “Preqneip yo “Tq “Woy 


LIC 
“PIT ‘bb “UDIDSIDIS UDILIAUIP AY “UOISSOIB9I Foul] Ul 9DUdIAZUI 


UO UONIITIs Japoul Jo yoedull sy (0661) “‘O ‘TESL 3° “WD ‘YotAINY 


"16-68 ‘17 ‘ajanbua p sanbiuysal ‘sayueuage sino[eA 
sap oAnoide | e uosdwioy]-ZzAIOH sinajyeunsy” (6661) “g “estINY 


ea 
‘TE “uDIUSUDIg ubIMaup aut (9P1d ZTEAB8L ‘OD) VAONYV pue 
uolssarsar Ul xUeUT yey CYL (8261) AU “YOS|aM 39 “O'C “UlseoH 


‘C8II-PLIT 
‘LQ ‘UOIDIIOSSp JDINSYDIS ubdIdaUp ays JO JDUANOL “IOyeUIT}SS 


ONLI SY} 0} SOANLUIO}S JULISISOI IOIINO (7661) “TI ‘ISOATY 10 “f ‘JOMH 


09-Lb ‘Sz ‘algnbua p 
sanbiuysa] ‘aysnqol asejeo ap sinayeunsy (6661) ‘dq ‘ousoyong 


‘T8E-9LE “L8 
‘UOYDIIOSSP JDIUSYDIS udIIAaUp ay) fo ;oUAnor ‘Sutjdues AdAins 


Ul ssoyeUNsa uoNBIqeD (7661) “AO ‘epwes 1 “D-f ‘oI[LAeq 


"QI-S1 6] ‘Sotujawmouyra] “UOISSoIdO1 
Jeoul] Ul UONeAIOSqO JeQUONYUI Jo uoNda}0q (4/61) ‘GY Yoo 


“VSHWVS be spueutsp 
ins 2iqiuodsip ““(WSHINVS) UuoneysIUTUpy sodIAIog Y[eoH 
[eIUDJA] puw asnqYy s9URISqNG ‘SddIAINg YIedH{ [eIUIJAl JO} J9}UaND 
Joy poredoid yoda [eo1uyos} je}saA\ “UBISAG AdAING SUOT}EZIULZIO 


wyeaH yewoyy jo AdAInNg g66l AL (0007) “*D ‘Alypnoyo 


“OUy ‘SUOS 2p AZTIAA UYOL : YIOX MON 
wing daaing fo sis{jouy “(€00Z) ‘f'D “UUDYS 39 “TY ‘srequeyD 


6002 ulnf ‘ayanbua,p sanbiuyooy 


“Ouy ‘suOS 2 Ad] UYOr : YIOX MON 
‘(4DUUTYS “- 19 SIoqUIeYD “Y ‘Spy) ‘Ping daaing fo sisAjpup suep 
[| enideyp ‘eyep Adains xajduios ym uolssaiSe1 oLjoUereduON 
‘(€00Z) “AW ‘Aoxysieag Jo “HV ‘Ueuglog “TY ‘siequieyD 


“690I-£901 ‘18 ‘MouDI20Ssp JooYsYDIS uDdIuaUp ay) fo joUANor 
‘uoyeulnss uoneindod sy1uy ysnqol IdIINO (9861) "TY ‘siequieyy 


OU] ‘sug 7 ABTIAA WYOL : 410A 

MON ‘(“JoUUDIS “dD 39 SIoquIeYD “Y ‘spq) ‘ojnq daaung fo sis{joupy 

suep ¢ anideyy ‘siajowered japour Suneumss 10} spouour 
poseq-jopoul pur paseq-usisaq (€00Z) “U'D ‘sHoqoy 19 “WC “epurg 

‘T6C-OLT ‘1S ‘anbusuvis 

ap ajpuoypusajut anaay “SAdAInS xo{dwiod Woy siojeunss 
yeuuou Ajjeosno}duiAse Jo saoueueA 94) UD ‘(€861) “Wd “‘Jepulg 


OU] “SMOG 2 ADTIA\ UYOL : H1OX 
MON ‘iuvauyjoy) fo saoanog pup vjoq joyuanfuy suidfyuapy] 
‘sosouspiq, uorssadday (0861) UY USIP 1° “A “UN “Wd “Aa|s|og 
“LET-LIZ ‘Og ‘aianbua p sanbiuyday ‘agsiyei9ug3 uoIssaisaI 
eR] Jed aysnqo1 uoneums” (7007) “V ‘ARTY 1° “4-¢ ‘UoUuNeeg 


a1ydeaisorqg 


‘g op uoleUMsa,| suep a}duI09 ud asud uoU }9 snuUOSUI 
ano ynod A Is swoW ‘g op s[epour ne podde1 sed soueueA 
PR] & (¢) suep sonbnsepeosoiajoy SooURLIeA Sap UOTNqLyUOS 
PB] JUIUWII}DIMOS djQ]JOI [I NO suas ne ajsnqol ys (q)a onb 
Issne dQUOW eI9d “apnNyIqIad DdAe SadUUOT]DII9S juos soyrun 
SOUTELIOO IS DWIQUI S9TJLI9A ySa Inb uolssoidxa ‘(q) “Iea 
= [(g)4] NJ suUOp sadueIedsa sao op uonmnsqns ey] 
“OXMAM |X -4=( ZZ) “gq onb sipuey “OXMAM /X 
= ()2'2°"'%) “q 389 suis Jorutoid np (¢) snos agpour 
ne yoddes red souvisdsa.] “gx — { ~ ‘a onb yuesoddns 
ug“ zau — iz'z'Z = (z—'2)(z-'z)""Z_—_ ousoy 
eR] snos saddojaaap ane jnod nor np some ey “(p) sueq 
“OPNIIO OdAe SagUUOeTaS sayluN sep UOU NO sUUdTUOS 
uoy[Queyos,| aonb “(¢) snos sjapour ne yodde1 sed sieiq 
sues yuoWoAneUTxoldde ysa (p) red guUOp mMayeUINsa J 

‘O19Z B WreIINpal as ueyd ne podde sed 
DOURULA OP IMI}VUIYSA, | ‘JUSUIOSUIDII UN,P SUIQIXE Sed 9] 
sue] ‘OpNyyIao suRS sa9UUOIOaI9S SaytUN,p aIquioU a] ‘”“U 
Jed g0e]dwia1 ane Wop uv 3a ‘Z op 39 (p) SUBp JURIMSIy SOUTWIOS 
Sop sonjoxd ang JUSAIOp spmyod s9Ae sagUUONDEI9S 
soymun soy :uejd ne yodder sed soueuea ek] JoUMso 
Inod sgjIpour ang Wop (g)4 op sNULOJ eV] ‘opmys9 DOA 
saguUodajas soylun sop juanU0S uo]]HUeYE | IS “apmytpso 
O9AB BQUUONIIPS 3S9,U gyUN suUNONeE.Nb yo ‘Mm kB dIeB9 
auypiqeqoid d9Ae dSIUIOI OAR SagUUO]THUeYDS JUOS soyTUN 
Sg] Is gidap [Nes uN e sseuUOT[HUeYD?,| SULP (Eg6I Jopulg 
Noa) uejd ne poddes red yuosIsAUO0d jsd IMayeWINse 19d 
ufxiam?'Z=Zz ya g’x — ‘y= 'a one 'x'’am = '7Z No 


() Wz (2 - 2)" Fv = pe 


ongojejeo ne X-[00-ZI N ‘epeued onbysnej¢ 


JUL} SUTUIOD IUTJap }S9 ¢ UOTDAS BP] B ad}UasSaId 
opm9,] suep 9SI[WN soURUeA op YoIMpuRS IMoyeUINss,] 
e'((a)3eIp =A 38 XM,X = W RO 22 WXMAM 
4 -W=(9) "yea sa ‘onbunduia opm, suep sion 
mayeumss,| “KAA ,/X (XA ,X) = g oP eepour 2] snos 
SOURURA BP “(¢) [eI9UaS s[9poUr a] sNOg “Japgd0id ap UOdey 
91199 op UONRoTNsnl eT 1o1 suosstnbsa snon ‘odAj-InaU9 | op 
S[NOTVS So] SUP OpN{IIIO S9Av S9UTIOTIDATIS Sa}TUN Soy SNOUT 
SUOAB SNOU ‘¢ UOT}D9S BI B 9ILID9p onbutdwis opnyje,| sueq 


3d Aj-1N9.119 | Ip UOHBUINSS | sup 
IPNZH.1ID IAB S9QUUOIII]IS S9}IUN Sap UOISN]IUT 


oxouuy 


‘Q[OIUe | JUS 
-J]QeISpISUOd JO1OI[aWIe Pp stued yuo snou Inb ‘syyorysUOD 
SOIIVJUSUIWOD SINI] Op SINIJeUTUTeXA Sa] 39 Joy Ud INd}ORpsI 
3] SUOIDIOUIOI SNON ‘Uonepunoy soudINg yeuONeN ez] 
OP Sa]]2o JUaWIDDI0} sed yUD}aTJOI OU 49 SIN|}Ne Sap Sa]][90 JOS 
gore] suep saoumuidxa suonepueuruiosel no suoIsnjouos 
39 SuonR}ESUOD ‘suorUTdo sj SAINOL, “[8OL190 ,U UoNUsAGnS 
B] Op NBI9A Ud sIU()-s}e1q Sop UOWepuNO, soUaTOS [eUOTIEN P] 
Jed sgoueulyy xnvary sop INS 9puoj }so [OTe JUasoid aT 


SJUIWIIIIAIUIIY*L 


‘IUSAB | SUBP SddIPNyd JUOIas aJonbua,p sasuuOp sap 
ashyeur | & Somejuswajddns sonbysouserp soysoidde .p uots 
-U9}X9,] 19 UONE\depe,T ‘soddeis us osu ev] 39 UONROTIFNHeNS 
eB] onb saya} ‘oxaduios a#euuoj[nueyss.p uejd np sonbysu 
-91)081e9 Sap a}dWI09 ITU} JUSATOp Inb sdueLIVA eB] Op SUOTLUI 
-S9 Sop JUsUATUOD ‘YOOD op souR\sIp ey suTWIOD ‘sonby 
-souseip sonbysyejs sanne.q ‘sonbidAye yuos soaneordxe 
SO|QUUPA Sap sINa]eA Say No sptod soy UOp sed so] 19}99}9p 
mod agstjqn ang jnad ja sasepuos ap sprod so] Jnes ‘osepuos 
op uryd np onbnsusjoevreo ounone ITUdAIOjUT yey ou Neadeys 
somjeul ey ‘saxayduiod ajygnbus,p sosuuop soy mog 

“syutod 
SUIe}I90 op UONeJapuoOd-snos ke] 9p NO UONLUTUMT]S | 9p SlayJo 
sop ajduioo juouue mb ssoyjodAy,p uoreoIJLIgA ap s}so} 
9p UONd9Xd | 39 DOURIFUOS op So|[eAIOJUI.P UOHONNSUOD 
eB] Inod souar1gyUI,p sainpgdo0id ap uOHLIOgRIZ,| 919 9[quIAS 
JURSSdIZJUT SYOIOYOI ap jofns un ‘ouod (7661 SULYZ ‘0661 
Tes], 39 YOIAMP{) ajayuenbes uorssaise1 vl suep yuosod 
as nb snuuoo uatq saurg}qoid xne a[qe[quies jsa suguOUgYyd 
29 (L007 FT) sasgpodéy.p UOHROYLI9A ap $}So} So] SUP 
soiasexe | odd} op Maue.p xnv} sop & jo [eUTWIOU xne ne 
DIMALIJUI JS9 DINJIIANOD LI JUOP DOULIJUOD ap So|[BAIOJUT Sap 


aBepuos ap spiod saj sed seigpuod JalAg] ap S}ayo yo Neadeyo soujey) : JURE, 2 17 92 


angoje}eo ne X-[00-Z1 N ‘epeueD onbnsneis 


qed yinpey as nb 90 ‘sayned doy odAj-mau9.| sp suoneumsa 
sop & nal] JouUOp ynod 9AdI9 4S9 IOIAD] Op JoIjJo,[ Wop 
SaJUaN Jul sUOTFRAIASGO sosnoiquIOU sp uoWoNbyeUIO;Ne 
ayjefei ib anbruvssut ampgooid oun Jastyn uepusdsadg 
‘QUIS 22 
AIOP [I.Nb 3a aJapour a] 3no} np sed aains ou [I.nb s]qQIssod 
\sd [I ‘sowignxa Y sinaeA op osned e yURLOdUTT IOIAZ] 
ap jaya un opassod jutod un Ig ‘arpsed as sioje yeunod 
JUDWIaSUdIaI SP SUOTJIPUOD Sep suBp aNoUIeIed uN IOWNSO.p 
asepuos op uryd 9] Ins sapuoj UONOU eT] UepUuddas ‘sIpIO9ds 
JUDUI9}N9I0N js9 yp yUouoidoid sjapour ay aonb yuesoddns 
Ud ‘ajapour UN Ins s9puo} sydoidde 9uN,p 3}x9}U09 9] sUep 
sojqeuuosie1 on9 ynod uoNelgpuod-snos ey] ‘soyjomiqeyut 
YX smeea sinatsnjd no oun spassod j1,nb sored yoynYd sreur 
‘spiod puvid un,p asneo e sed uou 9A9]9 ISIAZ] Op yoyo un 
oak ynad yutod un nbsind ‘fe1auas us ajUeSIeJsIeSUI d[QUIOS 
uolyelgpuod-snos ey] ‘ajapour np juowaysnfe. op juoW 
-191JUD IOUTUNI[S So] & }OS “IaIgpuod-snos so] e IOS JUD}SIS 
-uod suoydo xnoq ‘s0udlgzuT,] Inod 1a}1eQ soy JUSWIUIOS 
IIOARS Op a]J90 3S9 aJUeYOdUTT UOTJsoNb oun ‘soIfWUSpPI JOS 
puvis }SO IOIAZ] Op JoJjo,] JUOp syutod say anb stoy sup 
‘IOI SISN]]I SUOAL | SNOU SUTLUOD ‘SOUIQUI Sd] 
yusuaIessacou sed jUOS 3U Sed xNap Sd] SURP S}USNTJUT JOS 
mnb syutod sa] ‘sastjyn asepuos ap sptod say 39 ayduI09 ua 
sud a9 datop ueyd 9y onb no ‘sasiyiyn ang yuoAnad sameurlpso 
SolIed SdIPUIOW sa] anb ja a[qesousI jso aseuUOT]NULYOS_,p 
ued oa] onb ouuonnos uo] onb ‘sayuantul suotjeArasqo 
Sop Iua}u09 yUdANad ajonbua,p ssauUOp ap sa[quasua sa] 
‘gysnfe omeoul] s[apour uN Ins suONeAIOSqO.p odnoIs un.p 
no UONeAIOSgO g[NIs 9UN.p ajUeYOduT soUaNTUT,| 19}99}9p 
eB soguNsosp sonbysouseip sanbysnejs sap soajamuassa 
sajuRsodulod sap JUOS SNPISaI Sa] 39 IOIAI] Op Sjajja soy] 


uOISn]IU0D = °9 


(9002) JUeTTTe A 19 J Suep gisep [nes un 
B odeuUO][HUeYSE_p ued uN snos d9yndSIp 939 B ‘STIAAG 1° 
SV.LAGAC “rgnbus,p seguuop xne QDW Sep saieyuaurg|s 
sonbysouseip sonbnsnejs soureyoo op uonedepe.7 
‘sonourvied sop sojeqoys suoneumsa soaj ms uolyeArIasqo 
g[nes oun.p uorssoiddns ey] op jajja,] Jomsour mod snduoo 
SO “NPISOI 9] 39 IOIAD, Op 39¥J9,] JUoWAo1]dun spoduloo 
nb ‘yoo op sourysIp e] ‘ajdwioxe eg ‘101 sed suomioep ou 
snou onb s[yno sIoAIp JuesITHN ud [leap ua snjd saipme ane 
jnod UOIssaIZa1 BP] INS syulod sao ap soUSNIJUI, J ‘Npisei puess 
UN Jd J] QIe} IOIAZ] Sp jOJJo UN yUO sayne,p anb sipur} ‘nprsal 
ned UN jo 9AIIP JOIAI] Op JaJJo UN JUO SUOTJRAIOSGO SoUTE}IIO 
anb suoj}oN ‘sjutod sane say mod uonnjoses ey] Jaro1aure 
mod osiwo jso g{g UONeAIEsgoO., ‘mouayur onbrydess 
3] Suep ‘ UOTTHURYDE,| ap sjulod ¢/g sap uUNoBYO puaiduIOD 
moaudgdns onbrydess a] ‘snpise1 sap ate} e] e yUapuodsa1io0g 
S9ISap Sp & a}10Ip ve] e.Nbsnf syutod sap sayeonsea saouR)sIp 


GC 


SY] ‘sgouoy sutad sajo1so sap sed sgyuasgid yuOs 9A9]9 
IIAI] Op JOIJo & syutod sa] “¢ sms ey e gUosgIdoI so A Op 
SINJ[VA S9P UOTOUOJ Ud Sg SNOS UOIssaIs9I e] Op yURUDAOId 
sagisnfe smajea sop uorsiodsip op owueiseIp up) ‘sajues 
-IDAIP X op suNg[eA so] 1a}90}9p mod sasijyn ong JWsAned 
snptsai sat onb sored ‘uolssarsal op a[apoul np yuourasnie | 
suep sjuanjur yuos sjyutod sjenb Joutuuaj9p mod snptisol 
XNB SOUIQUIOD JUSWITJaMIIQeY JUOS ISIAI] IP S}ajo so] 


anbrydeas anbeyd suep ‘saduoy ‘suteaid 
$3[9199 sap ied s9juas9ida1 juos Sdd B®] ANS saseq 
JAI[9 AIIAZ] IP JoJo & szurod say ‘aso js9 BIg 
9U1Q1}X2 UONBAIISGO,| ‘ANatigjul anbrydess a] sueg 
‘sjuiod say sno} pusiduiod anatigdns anbiydess 
IT A=A & 99981} 3S9 99U9.19J91 Ip 2}101p By *X op 
sinajeA sap UoTdUOY Ua sagjsnfe sunaeA Sap 9d¥I] ¢ AINSI 


A 
000 O0I 


000 002 000 OSI 000 0S 0 


0 


000 OF 


sogjsn{e sinojeA 


000 08 


000 01 


A 


SOHSS wei SOP on SOAOS ier SOLS: oS049T 1 0000 


0 


sogysnfe sinoje A 


000 00€ 000007 000 00I 


660 97‘ I soynofe sjuoneg 
160 9I‘T SWT 
JAQ[9 JIA] AP JOJO & S}UIO 


s}Ulod sap UOeUIUII[a IAW = SA]. SNO} JBAY 
Sdd Jed suonewnsa 
xnev OOW Jed suoyeuinysa sap oney 


OHMS 8| 4nod uotssa139.1 Be] suep £00‘ & Anetigdns 

}S9 JIIAI, IP JdJJ2,| JUOP suOHeAIISGO sap UOHLUTUITI,| Saide ja 
JUBAR Sdd Jed 92 OOW Aed sasjguivied sap suoyeuysa sap soney 
¢ neaqe 


6002 uin! ‘ayanbua.p sanbiuyoe | 


“syutod suTe}I99 IaqUIO} 9sste] MOAe soide suoryeurss 
S99 & YC WD 10 %G I sSomougjur yuos stew ‘snuayas 
yuos sjutod saj sno} puenb sgqg sed suoneumsa xne % 97 
3p 19 % QT 9p samoauiadns yUOs QO $2] Jed suoyeunjsa soy 
“¢ NvIIqQu} J] ONSN][I,| BWUWIOD “yuRAe Nb 9AII9 IIA] Op Joyo 
B sjulod sop uoneurUys,| soide sayne sap saun say sayooid 
snd yuowayou yUOos Sgd Jed Ja QOW Sey Jed suoneumsa soy 
aonb so p 39 € xnea]qQe} Sop adesap as nb yutod anne uy 


Clk 660.1661 ,0 Irs lO 61'l sgqnofe sjuoned 


=P aigN 

p58 ¥0'8 69°89 86L7 767 (O8'I8 SH] 9P agN 
96°F SEO9PLI'I8TZ 89°€ COLES IS°6L6 I uIsI0 | & 
Sa9UUOPIO 


818 UOIeAIISqO.] ap uoIssoiddng (ul 


EL‘bp 170 001 IL'p 070 $60 sginofe sjuoned 


OP sigh 
CUE SLO MCS SL vest Ser L769 SH] OP 5,gN 
p9S IL‘ESE 98°E661 609 PS06h SS‘L86 Z SUISLIO,| & 
soguUOpIO, 

L00°0 


B INaLIgdns jSd I9IAI] VP 19JJ9,| JUOP suONeAIasqo sap uorsseiddng (1 
2 ‘1° woanpyje0g 2 “| jUIIDIJI0D sozuepuadgpur 


OOW 


Sdd jed uonewysy] sa sed uonewnys| sajquliv, 
OHMS #8] 4nod 

UOISSI.1B91 B[ AP SajguUIvied sap Sq Jed 39 QO Aed uoynewysy7 
p nesqe 


€r'Z 9L‘0 8‘ Os‘8I €1‘O ti 6 sgynofe syuaned 
OP sign 

819 PI‘el E718 SOIE £0 91'b6 SU] OP .,.4N 
PrOMe cL re S0viS (80 c 61979 CL 10C1- JUISLIO | B 
Ja) 010(0) 03 7@) 

1 -1-a ysDyja0y gS CUAIDIJ20D sajuepusdapur 


Sdd 1ed uoyewysy| 


OONW $3] 1ed uvonewysy| Sa{quie,A 


SQUUOT[MULYII S¥d C/g SAP 9}1[e}0) B[ JULSTTUN Us OHWS BI 
Inod uotssa.1391 B[ ap saajguiesed sap Sdd 19 OOW Aed uoyewysy 


¢ nealqe L 


“QSNJOUT Sd QI. aytuN,| puenb onb ojqrey 
snjd ang yUuoWaTjamjoe yermod odAj-moue., “Yuesyos seo 
YJ yuowssnle | ms yoyyo,p snyd yreme ji.nbsind 4nod snd 
JUIWII[QR|QUIDSIVIA IIIS NpIsei uos ‘splod pues un year 
818 gyun,| Is “onb suojoNn “p neayqe} a] suEP H0'g B € NeITqQe) 
3] SUP PIE] Op ‘Si] ap sIquiou 9] nod odAj-maUe | Jossteq 
}IV} UOISSTUIO UOS {(¢ aINBIy IIOA) Npisel puevsd un opessod 
818 UONBAINSGO, J ‘OOW Sa] Jed uorssaiso1 oun suep sed 9] 
JUDWIZTeSO S9,9 DUIWIOD ‘asepuOs ap sprIod sa] Jed so19puod 
UOISSdIZ9I 9UN sUep sadUISad sajuod sap sadAj-smoue 
So] Ins souaprour oun MoAe ynod julod nes un.nb Issne 
oNsnyl e[a2gQ “Sdq sed onb ODIW sa Jed jue} sanowered 
Sop suOT}eUIT}Sa soy suep sajqeidoidde sjuswiasueyd sop 


angojeyeo ne X-100-Z1 oN ‘epeueg onbysneg 


gsneo onbiun julod 39 aNjJoWO “p nes]qe} np (I) UoNoas ey] 
B sojuasaid yUOS s}e\[NSoI sa] ‘JUoUIONbruN UoTeAIAsSgO 3y)99 
SUIUNT[S MOAR soide uUOIssoIs91 eT aysnfe yUoUIaTe39 sUOAR 
SNOU ‘OWIQIX9 JUSWI9}SoJTULUL IS }S9 QQ JuIod a] aonb nA 
‘Sdd SN0S 18 QOW $2] 
Jed suorssoiso1 xne onbiydde sa onsouserp ap add} ourgur 
9] puenb mussmns yuaAnad inb syueyodun syese sap ajdurexa 
one UN.p ses |] “ONUIWIP x sop UOTTQUeYDE,p soURLIeA 
el onb sored ‘stwio yUOS QO S2] SNOS 9AgI2 IOIAZ] Op Jaya 
B sjulod sa] puenb juoWAnoayo squawIsne QOW se sed 
suOneUIss sop odAj-Ind1I9 | ‘aNUOD Jeg ‘Sdd SNOs uOIssaI3 
-91 R[ SUBP SOUTUIITS yUOS sWI9Nxa sprod un jueAe syutod 
suteyao onb yey ne np jso yeyNsal 99 (% ZL ap UONONpal 
oun) [70 & 9L‘0 ap assed ayja ‘saynofe syuayed ap siquiou 
a] mnod ‘(% 6p ep uoNonper oun) ¢/‘9 & pPI‘E] ap assed 
Sy] op ciquiou sj mod Sgq snos odAj-maua,] ‘ajduiaxa 
Ieq “SIWO JUOS 9AI[9 IOIAZ] 9p jOJJO & sjulod say puenb usw 
-9[QeIgpIsuod Issne yUoNUTUIP (“[-"q) sodAj-smos soy 
“UdIQ ZISSE JUSIOJIP sjulod sao ap UOISsTWIO saide jo yURAR 
soyIpoid smoayea sa] Juepusdas “2A9][9 IOIAI] Op JayJo & syuTod 
Sd] SUBS NO DIAP DIIJUIDA 389 soynole syuaned ap siquiou 
Ne 39 S}l] dp SIQUIOU Ne sag[auod yUOS sasuadap say ayjanbey] 
uojas sayeyyenb uorsnjouos ey aonb sp0s ap ‘saAneorfIUsis 
yUOs sajuad say ‘seo say sno} sueq yUoWIaANdedsal ‘o% Op 
op 19 % / Op yuenuIUMp squad sa] ‘sAaTA IOIAD] Op 1ayo 
e sjuiod ap aouasaid ev] & sayqisuas yUoWaTeSa yUOS saynofe 
sjuajed ap jo si] op soiquiou say mod Sqq suoneulyss 
SYJT °% 6S UOMNAUI,p jso UOKNUTUTIp ke] ‘saynofe sjuatyed ap 
siquiou 3] Inod ajuad ev] ap sed oJ sue ‘SOUTUNTTS JOS 9AII9 
JOIAZ] Op JaJo & syutod so] puenb (17°69 & 9I‘P6 OP) % 97 
UOIIAUD,P SNUTLUIP Sj] ap sIquIoU a] mod OOW sep s}10Ip 
R] ap ajuad ey] ‘aanisod moayjea oun & sAyeSOU IMITeA oUN,p 
assed ‘xneajqe} xnop so] suep dsaneoyiusis ysa nb “‘QOOW 
Sop oOIpP ve] ep oUuIsUO,| e e9UUOpIO,T (p Neo]qei np (1) 
U0T]99S) 9AITP IOIAZ] Op JaIJo & syutod sap uoVeuTUNS | soude 
JUDUI[QLISpIsUoS JUasULYD OO $2] Jed suoyeumsa soy aonb 
SuO}eISUOD SNoU ‘p 39 € xNRITGR} Sa] SUOIedWIOD snoU IS 
‘Sdd 
suOT}eUT}SS Soy Inod (E€g6I JOpUlg) YOIMpuULS IMoyeWNsSo,| 
ap UdXOUI Ne 39 JTJaMIQey OOW Sep sNuUo} e] sp uskou 
ne saguiysa yuos sadAj-sinaue sa] ‘Sdd Jed 19 OOW S89] 
Jed suotssalsa1 So] SUBP SOUIQUI Say sed JUOS 9U 9Ad[9 IOIAD| 
ap jayjo un yueXe syutod ap sayquiasua say onb JousI[NoOs 
ap JUSIAUOD |I ‘sIoJanoT “1000 <“Y sfenbsay mod xnoo 
JUOS 9AI]I IOIAZ] Op Joyjo & syulod say “g[g UONeAIOSQO,| 
JULYOWIO Ud (Il }9 JAI IOIAI] IP Joyo & Sed so] JULYOUIO 
ua(I Sdd ®] MS 19 OOW SA Ms sospuojy suoreumnss 
So] QUUOp yp NeIIqQe} ST ‘sed Saj sno} juRSITWN Ua Sdd 
RT INS 19 OOW Se Ins soapuoj sonouresred sop suoneuinss 
Sa] DUUOP ¢ NegIqQe} J] ‘soNoUIeIed sap sUOTRUIT}So so] IMs 
JUO S]I JaJJOo JonNb IIOA v a}SISUOD aJURAINS Ode}9,| ‘S9[999P 919 
JUO SJUANFUT JUSUaT[aNJUDA9 seo suTe}a0 aonb guUOP jue 


abepuos ap spiod saj sed sauapuod Jala] ap SjJaysa yo Neadeyo souyeY\ : JUBNI|EA 39 17 Zz 


anzojejeo ne X-[00-ZI .N ‘epeueg enbysneis 


sg.igpuod uou ODI S23 SNOs AIIAI] 
ap sjajja sap uolIUOJ Ua adepuUOS ap sprod 
say Jed sasgpuod AaIAa] ap sjajja Sap gdeay = =Z sandy 


OOW SNos JatAz] SOI q 


€¢ 


4 Ty _ 17 
gooey [yest SX] 


s9ynofe sjuaneg 


00°0 


Sl‘O0 6010 §=6S0'0 


Sdd SNOs J91A9] S}9JJq 


S70 070 


‘yuowonbiun QO oP epoujsut ey] 
red s9,09\9p juos Inb ‘g alle.| suep suduros xnoo aonb s3azja 
snjd yuomaAneoyiusis spiod un sno} yuo ‘yore, suep 
somis ‘uouonbrun Sqq epoyjou e] Jed s9\d9}9p 9AIIP IOLA] 
ap io}Jo & sjutod say ‘uawaysayrueyy ‘onbrydess np ][2499, | 
ye1ossnej o]ja,nb sored ‘astwio 3sd g[ Bg UONAIOSgO.TJ *L00°0 
op singeA xne sagde 939 JUO JOUdIIJOI Op So}IOIp xnop 
SY] ‘o9I9puod uoU UOISIOA INIT ap UONdUOJ Ua saI9puod 
IDIAZ] Op sjajjo sop uorsiodsip op ouTUeIseIP UN sUUOp Z 
dINBY VT ‘SOUONJUI JUSWI9][aNjUDAd SUOTRAIASQO Soy Jo19d91 
Inod sa1gpuod uou no saiapuod Ja1Ad] Op s}aJJo Soy WUOsTTHN 
sjt.nb uojas sayualajjIp SUOISNTOUOD sap Jol eB saysAyeuR 
So] Jousw yUaAned osepuos ap sptod sap sayjie, sa] 


“UOT[ULYIE | SUP UOTJLAIISGO 93399 Op 9UBI] ap OJOWINU J] SO UOIPAIOSQO.| OP CI,] :210N 


I Ty _ 1 
Ball yee Ox 


SW] 


800 =: 900 p0'0 Z0'0 00'0 
— a 4 . 
+ a 
+ 
El'vIZ1= 4x ESLe= 4X 
10 S‘0Z 
8‘0 O°! 
60 607 
60 90 
60 Ol 
L‘0 01 
8‘0 vII 
s‘0 SEZ 
€0 56 
9°61 8‘0 
8‘0 01 
s‘0 y‘LZ 
C07 01 
10 aS 
00 Bec 
rel ‘0 
‘TI L‘0 
‘91 90 
£0 LL 
L‘9 €6b 


LS9= M 
ZO 
6SI 
Z0 
SLI 
8‘9I 
S81 
80 
a) 
€T 
€‘0 
Cbz 
Z0 
r0 
8‘¢ 
r0 
oT 
Oe 
Ze 
re 
€‘0 

Mm / 1M 


splod 


7200 1S0'0 PSI 
€70'0 Z00°0 119 
€70'0 £50'0 9S1 
¢Z0'0 Z00'0 108 
7200 Z00'0 IIL 
970'0 Z00‘0 €19 
L70'0 L10°0 E91 
0€0°0 690'0 LSI 
0€0°0 Z10'0 LOZ 
re0°0 8r0'0 078 
y£0'0 Z00‘0 ZL9 
8£0'0 680'0 6LI 
Sr0'0 950'0 879 
S¢0'0 00‘0 0€8 
970'0 890°0 ILZ 
$L0°0 7Z0'0 I€€ 
SOl‘0 L10°0 99€ 
LS1‘0 S€0'0 OVE 
S70 L€0'°0 681 
68C'0 €1S‘0 818 
gagpuod “y OOW snos “y "sqo.1 2p I 


sjuej10dui snyid saj saigpuod Aa1A2a] 9p $}ajJa QZ Sa] JUBAE SUOTEAIISGG 


~ nesqey 


6002 un! ‘ayanbus.p sanbiuyoe, 


angojeye9 ne X-100-Z1 ,N ‘epeued onbysneis 


(@4ndt1gjUul) IInatigdns a98uv1 eV] suULpP Jorja. Ua 
SIU JUOS (Sdd Bl) OOW $2] UOTAS 9AI]9 S29 AIIAZ] Vp 39JJ9,[| JUOP 
sjuiod sa] ‘sajnofe sjuaned ap aiquiou np ja sy] ap a1quiou 
np uoysuoy ua sasuadap sap uorssadsip ap sowumesseig | ainsi 


spinofe syuoned op a1quioN 
000 Sz 000 SI oo0s 0 


0 


000 001 000 0S 


sasuodoq 


000 007 


soqnofe squaied op s1quionN, 
000 SZ 000 SI oo0s 0 


000 001 0000S 0 


sosuodoq 


000 007 


(9p0°0 = “Y ) ILZ Seo a] mod 39 (67,00 = “Y) TEE seo 
3] mod sg1opuod IsIAI] ap sjajyjo soy agua ajqeiooidde 
yes9 uN yUoWIaTeS9 9\SIx9 [] ‘sed yUOS a] DU Solspuod uOU 
IIA] OP S}a]Jo Soy onb spur} ‘spuess ana yusIIAe Ss Salgpuod 
IOIAZ] OP SJaJjo sna] ‘usAour sprod 9] stoj ¢]{ ap snyd yueyea 
asepuos op sprod un sajno} yUapassod [19 19 108 ‘IIL ‘€19 
‘TLQ SoqtuUN SaJ ‘sane sap soun sa] nod yuoJayIp sameyrxne 
SO[QULIVA SIMI] IP SINITLA Sd] IS SUIQUI ‘SoUgXS BBepuOS ap 
spiod ap anosap sayuelioge smmoyeA ap odAy anne uy) ‘sayqrey 
yuos sptod sa] JUOp stew ‘xnap xne no ‘sgynofe syuaned op 
SUI9XS SIQUIOU UN NO ‘s}I] Op SWI9Xd SIQUIOU UN e SoToOsse 
quos nb “pc] 19 OST “E9T “LST “O78 “6LI “ILZ soqtun say Juop 
‘so[qe[quas sed saQne Pp a}SIxo I] ‘s1apuod jajja,] anb pues 
snjd a109U9 3S9 OOW SI SNOS ISIAZ] ap JojjJo,] ‘ouuaAour 
RB] & MousyUI oseuUOT[HUeYdS,p sprlod un apassod seo 
99 onbsing ‘sgynofe syuaned ap 30 Syl] ap SaAo]o say somquioU 
sos op osneo e juowsediouud ‘spuexs snjd say solep 
-uod uou j9 saiapuod IaIAz] ap SjaJJo Sa] spessod gg aun, | 
onb suojou sno ‘saaiapuod sauuaAoul singq 32 Sa][ONpPrAIpuUt 
X OP SMIeA So] IQUS SATVeIAI oNTOSqe souRsIp e] onb sure 
‘UsAOUW S8epUOs ap spIod ne JanpIAIpul asepuos ap sprod np 
onel 3] ‘OOW Se] snos sosopuod uou IaIAg] ap s}ayJZo Soy ‘seo 
QZ seo mod ‘Issne sHUOU! nea]qR} 9D ‘jUsWIonbinduis 1]qQuI9 
L00‘0 9p neaatu 9] anb saaaze snd jUsUa[qeRIOpIsUOd sayTYO 
“68C0 & ZZO‘O OP JWOLIA IOIAZ] Op Sjajja soo op smoea 
SY] ‘spueis snjd saj OS Sdq SNOS IIIA, Op sjayJo soy 
soyjonbsa] mod suoyeArlasqo QZ So] uUOp Z NeoI{qQu} JT] 


Si] 2p dIQWON, 
00s I 000 I 00s 0 


000001 0000S 0 


sosuodoq 


000 002 


S}l] 9p 2IQWON, 
00s I 000 | 00s 0 


0 


000 00! 000 0S 
sosusdaq 


000 007 


‘SUONLUMT}S9 Sa] Ia}99]Fe WeLMod 
yuiod 99 ‘sowistuesio saqne saz mod sanuajgo saad ap 
JUDUIDUBTTL | SP JUSUIS}IOJ JUDLIOS saguUOP sINaTeA sas onb 
sore g (“| @ mouodns yeury sprod un juanbgsuos sed yuo nb 3a 
gsuod9l-UO0U ek] SP UONSSLI09 ouN,/p jafqo,] je} yO INb stew 
*OUTEHOO We} [ENIUT UOT[HULYSS,| SUP UONd9I9S B] JWOp ney 
snjd ssuuONUSU Sed Sap UN] Sd BIB UONeAINSGO,J) ‘7Z7‘Z B 
Je39 Jso od8epuos ap sprod 2] 12 (808 6L ‘ SOP 7: $ €°E98 6IS) 
=(sinofe * sy] * sre[jop 9p sierra us sesusdgp) “(1or g1g 
uoTeAIOsgo Jed dUBISOp) JUOWssI{qeja 399 Mog ‘onbrydess 
Np a][ayo9,] assnej jI.nb aored | omsy ey suep suasoidal 
sed jso.u mb juowdssiquja pueis son un juoWolesd 
dSIX9 [[ “| Neayqe) a] onbrpur,] swu0s sonbiysulAse zosse 
JUOS SoAeOITdxa saTqeueA Sap SUOTINGLSIP SIT “L000 2P 
]INIs SUIQUI J] JUSTIN Ud JorfoI US sTUI JUOS (Sqq) BdepuOS 
ap sptod say red uoyeiopuod re] snos 9A9[9 IOIAZ] Op Joyo 
® syutod say ‘seo onbeyo ap spiod ne sajjouuonsodoid juos 
Sa][nq se] sjanbsa] suep say[nq e sonbrydess sa] ouUOp sesue1 
QUIDIXNSpP eT] Jol[oi Ud sTUI JOS /00‘0 = 4/dZ ke moLisdns 
IS9 OOW SI SNOs ISAT] ap 19JJ9,] WOp syutod soa] ‘oeSuvl 
aignuaid ey sueq ‘(Woy snjd yWO9p yUoWIassI[qeia puLIs 
So] UN yURYOWIO Ud) SJUDUASSITGRID ¢$/8 op UOT[WUeYS9,| 
mod sjnole.p 32 sy] ap saiquiou sap uoQdUCJ Us sasuedap 
Sop uoIsiodsIp op souleIseIp so] oNUOW [| omMsy ey] 

oxouue 
ua a9guuOp jso (ued a] ms s9puoy sed jso,U 49) 9[9pour 
np puodap nb ‘amey ap uodey a09 op onbyeusyjeu UOTeO 
-jysnf ey op asstnbsa,J ‘Uawasusde1 un ke yWepgooid U0, 
IS DUIQUI ‘SOURLIPA dUN a1ONUd JUaTeINe aJUEd op S}USTOIJJZo0O 


eBepuos ap spiod saj sed souapuod Jaina] Ap S}oyo jo Neadeyo soujyey : JUeNIEA 39 17 Zo 


anZojejeo ne X-[00-Z1 oN ‘epeued onbysnes 


sa] anb 3a 9umsa 3so uonejndodiedns op s[gpoul un,nb 
J9PI,] DAB 9PIOOUOD IPNAIVIO DdAR SOQUUOTDIIIS SoITUN So] 
gmjouy ‘adepuos op uvyd 9] Ins s9puoy JUSWoInd UOTeUMMsSO 
aun suUep opMjIyIId OdAR SaQUUONEI[9S sg}IUN soz JOVEN 
mod atains aysoidde | e JUSWISWIOJUOD ‘aNJOXe Saf op Nol] 
ny ‘odAj-Inaue,| sp uoTyeUIT}sa,] suep | 9p sprod un juve 
SdIIUN Sa] SN[OUT SUOAR snou ‘jIns Inb UOISsoId9I Bl SURG 


98°8S1 9L'L Br trl I Splod 

808 62 90 ¢ Olvl sgcs 0 synole .p aigN 

SOV C £6 of ‘9 0 SH] 9P s9N 

CE98 GIS NTPSIT SOTO S'TEGTZ OO — (SIETJOP Op sso1]]TU) 

sosuodoq 

% 001 % SL %0S %wST %O So[qulivA 
sayguuend 


OHMS 8] 1nod uotssa1391 eB] SUBP SI[qULIvA Sap Sa~yUENgd) 
T nesqe Ll 
‘“asuodal-uou 
R] Op uoTooUI09 ev] soide [ op spiod un jworeAe sousiu 
-e310 / S| ‘no} uq “| B Mouadns 9809 sptod un jusTeAR 
sqyuepuodal apnjiyied ddAe SadUUOTJOIT9S sSayIUN SoUTeYIO 
39 npuodal sed 3U0,U [eu UOTTWURYDE | SUBP opNjITIIO SOAR 
SOQUUOTIOIIAS 939 JUSTeAe INb saylUN soy ‘sed sUTeYIID sue 
‘asepuos op uejd np ayens sed yuowlssedas sanjoaqjo 319 B 
mb asuodal-uou ke] ap UOT}D9I09 oun yUoUUSIdUIOD asATeUR | 
suep suosi]yn snou onb spiod saq ‘9g‘°gs{ e& [ op qwuRTye 
oYOMOo} ev] ‘a[eyUdW gyURS Op SOUISTURSIO Sop saT]Iey xne 
soloosse JUsIe}9 adepUos op sprod soy ‘[, dd uejd un uojas ou 
-UOTJOIIIS 9}9 B UOT[HULYOS | SUTUIOD “aTPUITUTUN 39 STeWTXeUI 
SINIVA SINI, IQUd JuROITUsIS edd UN Issne jUdJUaSoId 
saynofe syuaned ap siquiou 9] 39 S}I] ap aIquiou 9] ‘sasuadap 
Sop ][e}0} a] onb sowayxe Issne sed juatos ou sjI,nb usa1g 
‘9°91 “UMUNTUTUT 2] SIO} OOD OE OP Sead yneA Inb “¢‘Egg GIS 
op umMuIxew un apassod sasuadep sap [e}0} a] ‘adepuos 
ap spiod saz suduios A ‘uolssaiso1 e] suep sasnjouT saqq 
-PLIA Sop soyQuenb sap smoyeA soy oWIMsSdI | Nea[qe} IT 
“qquepuodap jo sayuepusdaput sa[qeLiea 
so] Inod yrenbueur ou Imeayea ounone sajjonbsap sunoeyo 
Iod ‘uotssaIs91 B] SUBP SUOTJBAIOSQO ¢/8 9SI[N SUOAR snou 
‘Te}0} NY ‘UOISsIs91 ke] Op sonowesred sop uoyeUNT}sa,| sup 
J[9poWl NP A SOULLIRA op sOLQeUI op aIda}uI sed suoAe.u 
snou ‘sojsAjeue sop yednyid ey red agXojduia ampgooid ey] 
ur] “AM ,X,(XM_,X) = 9 ‘Sdd majeumsa,] sastpyn 
SUOAR SNON ‘UONeIe[Sap op souuL,| JURINp saynole sjuaT]O No 
sjuoned op siquiou 9] 39 UOeST[NN,| nod jauuosied ua sa}0p 
19 SQ[[PISUI S}I] Op SIQUIOU 9] INS ‘sIeT[OP op JoT]TUI Us ‘ayURS 
dp sUISTURSIO UN.pP saje}0} sasusdap sap uUOISseISaI eB] 1SO 
QIU .P 9JEpour 9] ‘opnjd ayuasgid ev] sued ‘oMeOUT] UOISsoI3 
-91 ouN.p sanowesed sap suoneuMsa say Ins spuels juoUT 
-SANLIII 9OUSNTFUI 9uN MOAR JUSTeLINOd suUOTRAIISGO soUTe} 
-I20 aonb uslq Is ‘UOT[QUeYD9,] SUBP SodIPMd saTqeLIRA so] 
mod smgjeva op soyoydMo} sapuess ap jinpoid yuo ayes 
g1URS OP SUIOS 9p SOUISTURSIO Sop SddLIRA So[Ie} Sa] 


Lc 


‘(€00Z) JouUNYS 39 SIoquIeYD suep TUIFp se 
[I.nb Ja} FueUUOJUT JUSUDTTONjUSA Jso URTd 99 ‘oUOG ‘aI[Ie} 
apueis snjd ap saytun sap ua}U0d B ddURpUd} B UOT[HURYD9 | 
onbsind ‘uoneindod ey suep uonnqiusip ev] op aUSI9]JIp Bios 
uo][WUeYIE,| SuEp ddJ9puod UCU UONNQASIP Mg 19 LW PI 
® SodI][aI JUOS aJgnbua.| suep ssasATeUR So[qeURA op dIQUION 
"266[ S[IAIO oouUR | JUBINp saqnofle syusT}o/syuoed xnesAnou 
ap siquiou 2] snd 966] OP Ingep ne suIsTuUeSIO UN,p sjUdTIO 
/syuatyed ap siquiou 3] JUR}9 SWTWIOD TUYap ‘« sapostda »p 
SIqUIOU 2] Weg sdeUUOT|[HUeYSY,| SUP sPsTTHN (LL) Ie 
op omsour ey (QOOZ ATypnoyD ‘7007 UOSIOpUucH{ 39 
ployosiapuey) (Ldd) |[!@} e] & s]Jouuonrodoid gytTIqeqord 
ddAB OISOp [Nos uN e aes ued un uoyas 919 939 & sJgnbuo 
ay00 op uol[HueYyos,]T “xneisues xneyidoy sop 9s[ejuoUr 
QULS SP SUIOS dP SIDIAIOS Sd] INS Jd J[eJUSUT 9]UeS Op SUTOS 
Sa] suep sastjetoods soulstueSIO Sa] IMs SaT][IondeI 339 UO 
soguuop sop ayjenbry] yuemMp ‘g66] US sIuy)-s}eyq XNe dESTTROI 
(OHWS) suoneziuesi¢ weep [ewe Jo Aang By SUOSTTHI 
SNON “UOISsaIZ91 op s[dUIOXxd UN SUep IDIAZ] Op S}aJJO So] 39 
neodeyo soLyeul v] SUOUTUTexd SnoU “UOT]Des ayUaSOId BP] Y 
‘Sdd UOIssaId91 eT NO OOW $3] 
Ied uolssaisai B] dSt{WN Uo] anb uojas syuanJZur yUSTOS syuIOd 
sjuaigyip onb ustq yoy yeumod as [] ‘ronbijdde.p yuatAuoo 
[I.nb xnoo yuos asepuos op spiod say sed uoneigpuod 
e] Inod soysouseIp soy ‘ogsI[NN 3S9 Sdd UOISSoISSI e] IS 
‘ SOUTLURXS OQ JUSAIOP OOW S2 nod prepur}s sonsouseip 
sg] ‘sasl]Qn yWOS OOW say IS ‘Wouwssnle,p spowjeur 
PB] OSAP SIUOULILY Ud SONSOUSeIP Sop JasI[N yWe1Asp o)sAyeue 
un ‘UuOQde.p spour un 9suUUOp jue}9 JUepusdad ‘aT]eoI 
uoneoydde sun suep jUdIIJIP Sdd Pk] mod 19 OOW Sy 
Inod sonsouseIp sa] amsour ajjanb suep MOA op yuRSsaIDyUI 
yso [I ‘ouog ‘saudoidde yuos QOW se onb jo sasotpnonred 
suoyeoydde soap suep soloust ang juoAnod osepuos 
op spiod say yo aseuUO][HUeYDE,p ued 9] onb JUOIpuaTNos 
saysAyeue suIeyOD ‘suIOWTURANY ‘aST[QN oseuUOTTWUeYDE,p 
uejd op odAj np asneos e uonendod e suep onb owigur 
eR] sed 4so.U UO]]QURYDS,| suUeP X Sop UOTNqU}sIp ey puenb 
gJapour np o}doUI0OUI UoKeOIDeds e] anuOd uOToa}0Id 
op UMUITUIW UN ayjo adepuoOs op sprod soy sed uoneIapuod 
eR] ‘[ UuoNoIs ey] suep suOUUONUDW 2] snou sUIWIOD 
‘asuodal a]qeuea ke] & S9T[O1 ONO ynod ‘ouIQUI-a]]9 ‘Nb a9] 
-QUOD UOU ssuodaI-UOU sun yNpOUT « adeUUOT]HURYDE,p PUI 
-9Y49S »» a] puenb jnoymns ‘ajqissodumt stoyred yso asepuos op 
ued np soyqeLea So] S3yN0} IMs JUSWIDUUONIPUOD 9] ‘anno uy 
“soaneotdxa sayqeuea onb jue} ud sajUessolgjuI yUoWIONby 
-Qnuorss sed juos ou souteyis0 onb sored ‘ajapowl un suep 
osepuos op ueyd np sojqeura sop uoneisaqut.| e suonoefqo 
sop Jo_NULIOJ yUANed saysA[euR sa] ‘s9sT[QN a9 ned OOW 
so] Jed uolssoiso1 ey ‘Uo][MUeYOS,, onb Isute uoyejndod 
eR] mod youod jso [I.nb jo sseuUOT]HUeYDE,p eUaYOS 
9] JURUTULIDQ}9p osepuos ap ue]d np sofqeueA soy soyno} Is 


600z unl ‘ayanbue,p senbiuyse, 


QUUOT}IPUOS jS9 s[Qpoul uN puenb ‘opour ossoIH ‘salapour 
sap juswajsn{e,| suep Joynormed ua uonesapuod ry] 39 
[e19U93 Us osepuoOs op ued np sanbysuayoRIKO Sa] J9IOUSI,p 
wey af Joynsnf mod sgoueae ana juaAnod sjuoumsse 
Sop ‘[ UONS0S RP] B 9UUONUDWI SUOAR,] SNoU sUIIOD 


anbiigumnu sjdwiaxq *s 


A N'A 
[CS "X) FS CO — X) “N +1 = 
1S9 [CIDUIS S[QPOUT 9D SNOS UOTRAIASGO 1 PI OP IOIAD] 
ap 12.7 “4/1 yk = 4N 0 ‘4/'xX VX = “YT op suonew 
-1]S9 Sop yUOs sew ‘AV Jo *Z op osepuos ap uryd 9] ms 
So9puoJ suUOeUTSa sap snjd yuos ou “AV Jo “*y ‘ayeIoUas 
A oun suosyn snou 1g “4/349 —'K LAM |X 
SCOR AM <= ON AX MM" < = 7" )-SoA0 onbiTdde:s 
epedeid mb afjao e afqeiquieas |_ Ww Imod gnunojy su~ 


Ss Ss 
ie og eal Hoon ea at 
aj x'xim’¢ lay'xin’¢ 


Ss 
‘a/ xin” ¢ npn 


'X AM JX IMA JX 


I| 


I| 
< 


'X -AM I TAM UI 


JOAB 

AM ,X WX = H Jed aruygp suoye ysa neadeyo soujeu 

Re] ‘nuuod jso A onb 3a (A (0) ~ 3 a]eIoUDs soURLIeA 
ap sMjoms UN JUO JJapour Np MaLd,p SoULd} Sa] IS 

Ap icin are He no 


Neidio Rx iaetei M 


Mya v ant 
x 


jS9 UOBAIOSQO ,7 BI Op JOIAg] ap J0J9,] “(,9 0) ~'3 
“3 +'xq+0 ='d ‘omerrxne afqeued anes oun.nb yueuayu09 
ou a[dwis anesul] sjapow un suep ‘ajduioxa Ieg 
‘§ d0LyeU PY Jed 990u 
-Tua}9p onbiysur ej suep ‘x sop ‘“x ‘asIapuod suuaAoul 
B] op suUsIO[a say yso ‘x (Zz aonb no mM uadoui spiod ne 
yUSUIDATWeIedUIOD jNopMNs ‘pueIs so “M (| IS pueIs ano ynad 
IOIAZ] Op JaJJo un Yuanbgsuoo weg “xX e& ‘xX op siqoureyey| 
ap sourjsip ey sa (“x — 'x) _§ (“x — "x) W 10 “(g antdeyo 
‘SQOTZ B1aqgsIoA\ TOA ‘ayduroxe red) “x ke ayusd oplosdiyya 
un wuysp (“x — ‘x) _§ (“x — 'x) ‘onbyespenb ounoy eT 
art Py 


sf G@es x) -S (“x a x) N lg Winey aie 

Sd ‘FZ aaIJ9puod neadeys soiyeu ek] op [euOSeIpP 

JUOWIZ[9 J 2] NO “‘UOTeAIOSGO ,1 L[ OP ISIAZ] 9p I9HJO.] 
‘u/ mM! =M aAe MU = A onb jez Oy JURSTTNN Ua ‘sing 


andojeiwo ne X-[00-ZI .N ‘epeuen onbysneis 


“MA (Cixes xe idan xg 


~ 

= 
- 
cl 
lI 


N 
mfx kee Sue Sak) mn 


MA |X|-VX =H 
suoua}qo 
SNOU “SSUUONUSUISNS ISIDAUT SOLYeU ke] ap JURAIAS snou 
uo neodeyd some ey jueyyduns uq ‘(| — d)x({-d) 
SUOISUSUNIp op aoljeur oun ysa AV (38 t— INKS 
19 ‘T x ([ — Z) UWolsuouMp ap 1n9}90A UN sa AV *Y = AO 


1 0 0 
(hy SS a N | = 
I x 0 pak 
I 
aS x4 Si 
Suri & 98x ober mn 
S y S77 
= Vv 
yen N Nee SANS | 
Sap ee ieee acuta 


‘souuoniyed somyeul 
oun Pp SSIOAUT] JURSTTWN Uq ‘([ — d) x ([-— d) suotsuaunp 
op somjeur oun jsa 'w jo fey 9% — Ky yuos s}USW9]9 
so] yuop [x ({—d) uorlsusump ap mo}ooaa un yso ¥ } NO 


'v *y ‘XM 7X TMX ux 
is J-(" J=cxoa| |=¥ 
DN XM /T IMT yl 

SULIO} 

RB] SNOS agTNIyeO ysa YW soLYeU eT“ ([ — d) x uw suoIsusWIp 

ap souyeur oun 3sa 'y 49 ‘T & xneSo JOS sJUSUIZTS Say sno} 

yUuOp | x U UOISUDUMIp op ma}d0A uN yso | ‘(| — d) x | UoTS 

-USUIIP ap smajoea sop sjuasaidar (7 "x “-+ Hx) = |X no 

“x WMI 
eta ox OCC NK = oe oo = X 
ig Ad 

anb ayos ap ‘a}Ue}SUOS 9UN JUST}UOS 

[i.nb ya (7) sa [reaAvy ap sJapoul a] onb suosoddng “9}1un sun 

mod xX op smajea sap jo spiod np sjayja say yuoredas mb 
S}UDUIE9 Ud S9soduIOdap aj9 WANA IOIAI] SP S}OJJo SIT 


AIIAJ] VP S}9JJo Sap UOIsodwi0s9q, “fp 


‘aseuUoT[HUeYE ,_p ued 
ne sogysnfe ang ke a1gTUeUI ap SaestTeIdads JUSIOS SDURLIVAOD 
ap SALUT Sa] NO saguMysa sadAq-sMae sa] onb yussgmnbe1 
Inb ‘yoo, ap soueISIp P] J][9} ‘SONSoUseIP xne JUSUOTTeNUOD 


aBepuos ap spiod saj sed sgugpuod JalAd| Bp S}ayyo Jo NeEadeyo soujey : JUeINIEA 39 17 0Z 


angojeyeo ne X-[00-Z1 oN ‘epeued onbysnyes 


‘sgidap smorsnjd e no gidap un ke ‘a1yHeNs os sdepuos op 
ued 9] onb yuonbrjdde,s a8epuos ap sprod say red soigpuod 
IJIAZ] Op Sjajjo sop 39 neadeyo someul ev] 9p sarnuLOj 
sa] onb suojON “xnap say no 4ueueqe sprod un ‘sayuBLIaqe 
X op singe sap sed gsned ang yned JdIAg] op Jo[Jo pues 
un nb onbrput saiapuod JatAz] ap s}ajjo sop UONTUIJap eT 


ly Foe mae way xin = My ln (0 
'0=M_,H/X-M,_,X =(H-DM_X @ 
‘MH = M/X_VXM = HM ( 


: SoyURAINS 
‘IODA & SoTloey ja ‘sayyn samejyuows[ddns sayaudoid soy 
apassod Sgq neadeyo soujyeu e] ‘ano uq ‘o919puod UOISIOA 
mg] red sagoejduial yuatos sagigpuod uou neadeys sooujyeul 
sa] anb uonipuos e (¢ anideys ‘ogQz [[eAoy 19 URUTIOG 
yuenyeA TOA ‘otduroxa Jed) sagiJuga yUd\Sel Z UOT}IES 
eB] & sagouous (IA) & (It) sgygudoid say “(xneSout sprod 
‘g]e19U93 snd oJoTUeUL ap ‘NO) saTeSoUT UOT{DITIS Sp SazTTIGeq 
-o1d o3ae aseuuO][nueYse p suejd saj nod anbrynsurAs snyd 
yso.U Sqqd neadeyo somjeul ey ‘oo1opuod uou neadeyo soy 
“PUI PB] & JUOWOTIeQUOD “AA op Noy ne | AAA = (AA JOU 
-g[ duis juest{yn ud A JOISNUI.p UFR syeI[NSaI So] JolIpOU 
ap ajqissod jsa [I Guepuedas “uoIssoIsa1 PT SUeP A SOLNeUI 
oun sed sist ,u aysAyeue | onb gsoddns js [I ‘uonepNUOy 
ayeo sueq “M'X Wx = “y jUos neadeyo somjeul ev] op 
gjeuoselp ey Nod JatAg] ap SJayJo SY] “KAA XK = W o2AeV 


MX ,-VX = MA |X,(XM ,X)X = H 


Jed 
SIUTJOP }S9 39 adepuOsS ap spriod say ynjouT neadeys soLjeUl 
eR] nO ‘KH = A Jouuataap soyipoid smMayea so] ‘UOIssoIdaI 
eR] suep a8epuos ap splod sap ajduroo nud} jsd [I.S 
‘AM /X_(XM /X) = 9 
‘sorgpuod soled sorpuloul soy red moayeuyso un.p suLoy 
eR] Juanbesuoo Jed erpucid g (Sqd) edepuos ap sptod say red 
srgpuod moyeumtysa,y 19 (1) B IMpar as (¢) agpour >] ‘| = A 
enb suosoddns snou If “X,AM ,X,(X AM ,X) 
= g§ SUIWOD JUOUIO}IN1TTdxa sanjosal aye yUaAnod suonenbe 
sad) ‘ory uonefndod ey suep a]j90 op yUoWIZ]qQe[quIOSTeIA 
UQIIP uoyHueyos,| suep { op uoynquysip ey jonboy suep 
jJneuuoju ueyd un,p adw109 U9} nod AW, Suep sastyyn 
yUos ‘UOISNOUT.p sayiqeqoid xne sjauuonodoid juoW 
-OSIOAUT JUSUATJOM|IGeY yUOS ‘sayst{Iqeqoid suOT[HURYOS so] 
suep ‘Inb ‘a8epuos op spiod say (“mM “ ‘n)3eIp = AA 10 
(“a “la) Seip = A ae “QO = (PX — A),_AM |X UoneU 
-1S9,p suonenbs.p sutasAs np uonnyjos ek] Isa _ J op (AWdd) 
souR[quIosIvIA op UmMUTxew-opnesd np moajeumso, J] 
",O ayuR\suoS ve] ped eB anUUOD }so INb ‘0 ‘a goURLIRA ap 39 
(Q suUSAOUI op s[eULIOU IO] dUN JUDUTUepUadspuT jins ‘3 no 


1 


(¢) (,o'4 ‘Q)N pul~'3 “3+ g x= 4 


61 


: SOXTJ SJaIo 
B SIILAUT] B[9pOUur UN jsd yUDde/-snos JaIMjonNs sJgpoul af aonb 
suosoddng ‘asepuos op urd 3] Ins sospuoy sapoyjou sop 
jUeSTTIN Us 9UITYSa sIOTe Jso INnb uOHe[Ndod ap [e}0} op odAy 
UN jSd UI}SAS 3d “d9AIOSGO }1e}9 SOUS sIUTJ UOnRTNdod 
eR] Is gjapowr un mod ypespuatAuod nb uonewnsa,p 
suoyenbs p oulajsAs 9] JOWUOJ & a}SISUOD ode}9 oIOTWOId PI 
‘QoUR]qUIASIvIA Op UINWITxeUI-opnasd np syooidde | sueq 


asepuos ap spiod soy 
Jed a919puod neadeyd syne °¢ 


‘soyueliage soqyrun .p 
Q0Ud}SIX9,] Op dANaid eI Issne SUUOP ISIAZ] op s}ayjo sop 
onbunduia uorynquysip ev] suep sjanjiqeyut seo sonbyjonb jo 
seo sap pednyd ey anus yueyodum yre99 uN,p 90U9}SIX9,'J “(7 
anideyo ‘Og6I “T]09 12 Agisjaq) “4 /d = y ‘ouusXou RJ sIoy 
xnap B [B39 SUTOUT Ne JSd aTIO IS 9UIQNXa SUID daI9PIsUOD 
JUUT[OMIgGeY 3S9 “Y [ONPIATIPUL IOIAZ] Ip 39JJO,P IM]eA oun 
‘puvis JUSUITQeuUOSTeI SddUUOp op squIasUa UN sueq 


upx"s 
=X MOT (KX) OW CX) es 
coda pe COMMA I 


: SO9IJLIDA JUOS Sa}URAINS 
sqaudoid xnap say ‘ayuR}sUOD oUN yUAUOD ([) BJapour oy Is 
‘u/d = Y 4Sd UdAOW JDIAI] OP 19JJ9,] 
anb onbijdwit mb a0 ‘d = (x)3ue1 = "y'% (A 


Ah GesOapeal 
-0=X(H-D 0 X = XH (In 
°0=(H-DH 


no ‘“,H=H_ 10 ‘quujjodwiapt jso AL” (Il 
6 ul 


<“y = “y no ‘anbinawiisjsa C(I 


: Sayyn 3a sayeroads sajaudoid sasnaiquiou ap yuapessod QOW 
SJ] INS saseq IIA] Op sjayjo say] yo neadeyo somjeur ey] 


“X Op MIRA ke] Op UONLUILLLID}9p e] SUep 
jeronso ajor un onof ‘x ‘{ ap oyooidde,s "y 1g “x “y !Z 
+ x"y = 'x"y!Z = ‘x onbsind ‘aaisnfe moyea aidoid 
es ms ‘x ap ayja,] amsawi mb ‘neadeyo soujeur ayo ap 
Q[euoseIp B] IMs JUSUIZID 4! 9] JS9 “XW |X = “Y “IaIAg] ap 
PH. (SL61 YsfomM 39 ul[seoy * 7 anideyo ‘OgET YosTeAA 
39 yny ‘Agsjog) Aoyny red soy arotuaid ey mod astjyn a9 
® wou 92 neadeyo sone ey gjedde sa ~X WX = H 00 


‘AH = A|X,-WX =9X=A 
juos 
A Sa9Alosqo smmoyeA xne yuepuodsau0d XA soaysnfe smmayea 
SY] ‘I[GISISAUI J9 JALIRD sOLeUI oun 189 X .X =V 0 


(2) ‘A X,-W =A X(X,xX) = 9 


6002 uinf ‘ayanbua,p sanbiuyoe 1 


1sd._ J OP (OOW) SoueuIpo seed sarputou 9] Jed 
uonewnss,| “duel utojd ap iso KX doLNeUI eT onb juesoddns 
uq <a Gales Ig) rs 10 7G Gowe ‘Iq) 4 d dae (POF wx) 
& 'y JOAPR (“x Cove ox) al ox Ag Coes 7 #) e. 5% no 


(1) 12=(3)A 8+4dX=A 


}SO [IRA] Op IITeOUT] BJQ9pour a] aNb suos 
-oddng ‘ouuioj es & yey JUeAR S}UDWIOSUeYD Sanne .p no saat 
-eojdxo sayqeuea op uorssaiddns ‘saaneordxa soyqeuea op 
ynofe red o1puojsoidde snjd uonenyeas oun saide dstAai ang 
ynod [J ‘ayjamdsouos uonejndodisdns sun xno 93] yep 
Inb amos ke] jUe}9 SUIWIOD 9ysATeUR UN Jed yUoWIoNOSTAOId 
Mopisuos yso nb sjapoul un 48d pipAD.Yy ap ajapoul ul) 


ODN $2] Ans sapuo0j nvadeyd sdLyey, *Z 


“samreyuouig{ddns aysiayool op xneaey Sop opuaid yuotes 
-mod onb suonejusu0 souteyI99 suouUOp jd sje}]NsoI sou 
SUOWMSII SNOU “UOT]D9S SIDTUWIAp L] Y “sTeJUSW gues Op sUTOS 
So] Suep sosITeIsads souistues1O.p “gudap un & UoOTTHUeYDE 
un ims jueAndde,s onbuswnu ajdwaxa un suouuop 
snou ‘¢ uondas eB] Y ‘soidap smoisnid e.nb Isure adap 
un & asepuos ap suejd xne yuonbrydde.s ajanbua.p sasuuop 
xne suoIsua}xd say ‘jUeYOdUTT IDIAZ] op jaya UN Japassod 
qusanod syutod say yuouT0 anuoW Inb uonIsoduosep 
oun.p onb sure ‘asepuos op sptod say sed sgispuod 
INAZ] IP sjayjo sop 39 neodeyd soujew eI op suojeEN 
snou ‘p 39 ¢ suoTNoas xnYy ‘saj}oUdoId smo] ap souTe}I99 39 
IDA] IP Sa]Jo So] ‘SdITeUTPIO S2LTed SIPUIOU Sap apoyo 
ey red onuajqo neadeyo soLjeul ve] SUOALID9p snou ‘7 UOT}das 
RT YW ‘ys | sumI0d ag]UasaId jso d[OIe,] Op ayins ey] 

; “S[INDS SIMI]PA SOUTEYIIO ap aseq FB] Ins syutod say] yUOW 
-onbyewojyne yemypoxo nb ampsooid oun Myqeid.p nary ne 
‘uouajsnfe | ap sNjOXd a9 JUdAIOp sjutod soy Is BIOUTULID}Np 
aysAyeue Un ‘adUaNTUI IMI] op suOsTeI so] yUOWTUOprud sou 
-TWeXxd 39 S]UaNTJUI syutod say] 9[999p IOAe soide nb suowms 
-g1d snou ‘ajome yuasoid a] sueq ‘a[9pour a] Jaysnfear 49 
Jowuuddns sa] ‘syuangur syutod say 19}9939p mod saonbrjdde 
jusTelas sa[da1 souTeyI0 ayjonbe] op mda ua onbynewojyne 
ayooidde oun Ja10QPI9 & jLID}SISUOD J]]IINeU UOISUd}xd SUL) 
“Sd9NSIZIIUD JUSUID}IAMOOUI SdgUUOP ap 3Ud}[NsaI sayueIgR 
sIngjeA $d Is gsuas }sa INb 99 ‘ajapour a] Jaysn{e1 ke 39 
SHUBLIDQE SINIVA Sd] JOUTUNTTD & d}SISUOD ‘dATeU O.N9-jnNOd 49 
‘QJUSPIA UONNIOS du) ‘snsiquie nod onbjenb juos sa9}99}9p 
29 UO salja,nb stoy aun sayUaNTUI sUOTFRAIOSQO Soy I9}Tey 
op uosey e] & yUeNb siNjeID}II] Be] SURP SUUOP S]IasUOD Sa] 

"SayUPSOdUIOD SUMUIOD SNPISAI SP 19 IOTAZ] 
OP S}IJJO Sap sloj eT] & Japassod yuaIDAe,S “(116 ] YOOD) YOO 
op d0UR}SIP asnowe} ev] onb saqja} ‘sonbysouserp sonbnsne}s 
SISNIIQUIOU ACJ “SNPISZI SOT IDAV SONTRAD JOS S]I,S SddVdI]JO 
snjd yuos 3a onbysouseip oui9s oun .p oped yuo} JOIAZ] Op 
S}QJJo sa] ‘azenbus.p sssuuop sop saide.p sysnfe sojoepouw 


ongojeje9 ne X-[00-ZI N ‘epeued onbnsneis 


Sg] Inod sonsouseIp op dpJ0s ouUN.p jUL}TIeEN SdOUDIOJOI Sorel 
Sap 9UN,| 1S9 (6661) PIeqneIH 39 Woy ‘adepuos ap sptod sz] 
red a9g19puod uolssaiZ91 eB] Sp 9)X9jU09 9] suep UONRIDIdI9yuT 
INJ] Sp 19 UONRSI[NN Mo] op UOIssnosIp ouNONe ainNjeloyT] e| 
suep 9ANOI} su UO JUEpUsdad *99)Nd9x—a 4sa saJapuOd saueD 
SoIpuIOU So] Jed uoIssois91 sun puenb prepurys sja1orsoid 
so] Jed syinpoid xno9d onb souigu say ywWos suojuasoid 
snou onb JdIAI] op sjayjo soy 39 neadeyo somneur ey 
"saguuop op sjutod sutey9o ap sjajjo say Jonyead 
Inod oysouserp op adAq un yuM0}j ]I,nb 39 ‘saguuoNususns 
SuOSIeI Soy Inod yUOWAT[aMUDAd ‘s[Qpow uN,p JuaU}snIe J 
suep odepuos op splod soy Josiyn.p aploap ev aysdyeue 
un.nb suosoddns snou ‘oponre juasoid oy sueq ‘azonua 
oruy uone[ndod ev] op yWesodsip uo] Is nuajygo yesos nb 
sisnfe XNdIWI d[ dIeOUT] B[9pouw! og] Inod steiq sues yUOWIOAN 
-eurxoidde juos sagigpuod suoneuinss sa] 4Ip juswanny 
‘JUDWIISUIIOI Op aNoWeIed UN dalOoUd dUIISd osepuOoS op 
spiod sa] red agsgpuod uolssois91 ev] UOUD}990LI09 dIJIDOds 
sed jsd,U d[9poul J] IS ‘ANNO UY JHeULIOJUI }so UOTTHUeYOI | 
aonb a3dwi0o mua} sed ou op ye} np JoayNsor yeunod nb 
stelq d] aNUOD a8q01d asepuoOs ap splod sap uoTesINN,T 
"¢ UOT}OIS BI B SUOALID9p snou aonb onburdusa 
opnd,| IP I]99 & IJqQe[quias so UONeMIS add ‘UONe|ndod 
eR] Suep uONNqmsIp es e wodde1 sed ay0Ip ev] SIDA Jg]eI9 
eI9S UOT[HUeYII | SUep X dP UONNQMNsIp x] ‘x e DOTTOI 4SO_X 
anb 49 d{]Ie} Ind] Sp X aIMsoUT SUTe}IOO OUN kB SaTJaUUOTIOdoId 
sayiqeqoid sap s0Ae saguUOTda9S yUOS UOT]WUeYDI,| op 
SHUN SJ IS “(E71 Wondes “€QNZ) AOYYOIOAS 30 URUIOg 
‘sioquieyy sed suUuOp jso uUoneMIS a90 op osduoxa 
uf ‘UoNrndod e] suep UONNgMsIp ve] op sap UOTTHUeYDA | 
suep { op smayea sop uoNNqmsIp ey [anboy suep /youuofu 
asepuos op ued un e& uo] onb sioje Wp UG ‘adepuoOs 
op urd oJ ms spuojy ‘edo1 oseuUOT[HURYDN,| Op suds 
ne jd IeJA J[apour ne yoddes red stoj eT & steIg UN,p soyoeyUS 
JUOIas sanouered sap saispuod uoU smoayeWNsd say ‘SoATIeS 
-1]dxo sajqeueaA sap smoajeA sa] IMs JUOWOUUOTIpUOS soide 
suou ‘X & SOI]a1 ana,p sojqudoosns smajoej ap puodap 
aseuuolyueyoe,| Is uepusdag “K soaneodxo sojqeuea 
Sop sinajea soy yueyoes X op sjuepuodopul juslos asep 
-uos ap spiod say onb juargInbor sanbnaujodAy suonipuos 
So] ‘Jaynonred ugq ‘sjapour ne yoddes sed steiq sues jUOIOs 
UOISSaIZaI op sanouresred sap sasigpuod uou suoreUrTso 
So] SIoTe ‘ajapour a] suep sasudwios soaneoi[dxo sojqeLrea 
Sop juowenbiun pusdsp oseuuoynueyss., onb 49 yUOUID} 
-991109 gIflo9ds jsa aMedUT] BEpoul 3] IS “KX soaneo[dxo 
S3]QVULA Op a[QuIOsUd UN Ins dgseq X ssuodo1 oun orpgid 
mod ones] sjapow un,p juowaysnie.| op 3}xX9}U00 9] 
suep 101 (¢°7'9 UONDIS ‘¢QQZ) JoUUTYS suoserydesred snony 
(E79 WoNdeS ‘€00Z) 
JouuryS Jed yuourajoulsons soumNsol yUOS seu ‘s}TENsqe 
yo sonbynewsyyeur zasse ang yudAnod sjlejop soy “onu0 
39 Mod syuowmMsse sa_ yUasodxa (9 anideyo ‘6g6]) WWWIS 


aBepuos ap spiod saj sed sasapuod JaIAg] ap S}ayja ja NEadeyo soujxeyWy : JULIE 39 17 QI 


‘THLOT GW “ed 2921109 “RH ABAPT SIZ ‘PuejArep Jo Ajisioatug) ‘ASojopopop] ADAING Ul WeIZO1g 
jUIO pue “URBIYOIP] JO AjISIOAIUL) ‘I9}U9D YoIvssoy AdAING ‘JuURTTTeA PIeYySY ‘O¢S8O7T GW A[!AYooY ‘pseasjnog yoreassy C9] “eIsom “TT nyzuell “| 


19 (9661) UUEULIAyad “(pp “Ep SUONdeES “666T) PIEqneIH 
9 Woy “(T anideyo ‘EQQz) JoUUTYS 19 sIoquIeYD “(O'TT 
‘SZ LT] SUOTIOAS *EQOZ) AOYYDIOAS 19 URWJIOG ‘sIoqueyD 
‘(¢ amideyo “¢Q0Zz) SHeqoy 39 Jopulg TIeIgp ua UoNsanb 
990 ap juaTeN (€00Z) JoUUTYS 19 SIoquIeYD op 19 (6861) 
[J09 19 IOUUTYS op UONdATIp eR] snos sorjqnd sjlondo1 sx] 
‘Sa[apoul Sop yudWIg}sn[e | suep adepuoOs op sprod soy JosITHN 
UOU NO yNej [I,S MOARS ap a][99 1sd aJonbusa,p saguUOp sop 
saysAyeure So] juasod as onb suonsonb sa]]ou19}9 sop oun J 
“xNe}0} Say nod 
sojuRuioge sinayeA xne sajsnqol uonewmss,p sonbruysa} 
sop JwaIpm? (6661) Ausayong onb sure “(g66]) MeyouoY 
12 YSPPM ‘(Z661) SOATY 19 1OMH “(9861) srOquIeYD 
‘sqyuonyjut soytun say no saddeis say Joz9puod-snos mod 
W uoneumnsa,| Ins saspuoj sarsayens sop ywostyn (P7007) 
IARTY 39 JUOUINedg onb Isule “([QOZ) WE 19 JayxUSYIS 
‘AYSARTSEZ “DOUSNIJUI,P UONSUOJ Op SULIOJ ouUN 39 Sed ap 
uorssoiddns vy] Ins saseq sonsouselp op UoNeSUOUSp P| Iey 
(L861) Wg ‘Uorendod op ]e10) uN, p uosdw0Yy | -Zy1A10H] .P 
IMgeUINsSd,[ Ins sajueLaqe sMIeA sop 39J9,[. op uon 
-sonb e] yUapsoge (666) OpreydIg-zounyy 32 SoADY-ZouNPY 
‘o]JOGeyY-OudaIOPY 39 (S66I) JOSITNH ‘sayduns saanduosap 
sanbysye}s sonne.p jo uoneyndod ap xnv}0} sap JouWNsa,p 
}So JOa[qGo.| puenb sawigyxod adepuos op sptod soy Jonseyo.p 
NO JasI]eoO] 9p SUdAOUW sUTeYDD Op JUDINISIP (€66I ‘0661) 
IMNOd 12 (7661) [epwBS 19 JIIAoq ‘SaMeputy uorssaisor 
ap sajapour soy mnod sonsouseip xne uoyUsye,p nod onb 
opi0soe .U sTeu ‘ajgnbua.p saauuOp sap solide .p suonewmso 
SOUTRHDO INS saUBLIDge SINICA SOP 39JJ9,] Ip suOTIssndsIp 
Sop oYjJo sosepuos soy IMS djURISIXd sINjRIONI] eT 


‘asepuos op sprod spuei3 
So] Jed Issne steur ‘souiaXd KX op SMa]eA soy Jed yUsUIaTNIs 
uou sogsned sayUonyUI yUdWIaTJaMUDAg ~ sUONeAIaSqO 


So] Iqj09}9p nod sayyn yUos ossepuos op sprod soy 
Jed soigpuod IdIAd] op sjayjo soy 19 Neadeyo soujeUI P| ‘UOT 


‘soigpuod sgued solpulow ‘ ajJgnbus,p saguuop ‘ d1legUl] UOISsaIBaI $ soUaNTU] 


snjd suonuoUl 3] Snou SWIWIOD ‘ou9g|qQo1d UOs mod jUdTAU0D 
asepuos op spiod say ITUSAIUI jUeSIey IMayeuNsd un,nb 
aploop e aysAyeue,| onb suosoddns snou ‘aponre yuasoid 9] 
sued (€861) Jopulg op segpi sop juendsur,s ua (¢ anideyo 
6861) WMS 19 YOH “ouuryg Jed ayLD9p ‘souelquios 
-IeIA op ummutxew-opnosd np spomjour ey] sed sagnojes 
JUDANOS yUOS saNoweIed sap suoneUMsa soa] ‘saxa;dur0s 
ajonbus,p sosuuOp sop juesI[yN ud oMeoUT] UOISsoIdoI 
eR] sued ‘I®IA J[Qpou np souMsoa sanourered say JoyIe9,S 
yuoy nb syutod say Ja}990}9p ap saysAyeue xne smMoULIOd 
JUIIBIADP JOUANTJUI,p soNsoUseIP sa] “« IVIA »> JUOWIITTIEI 
}S9,U dsJa9poul UNoNe nb jueUsANOS 9s Ud ‘aANOIpaid no 9An 
-duosap souessind oureyi99 oun juepessod s,opour un IdANOg 
ap JusUaTjanjIqey 3So JNdA[go,] Guepusday *« TeIA » s[Q=poul 
3] suojayjodde | snou ‘ay1pouru0d ap Ionos Jeg ‘uoNendod 
eR] op atued omofeur ej mod ysnle ustq yUOWaTQeuUOSTeI }I10S 
gjapour un.nb suosoddng ‘uonejndodisdns op ssamjodAy 
oun jURVOUID Ud s}INsUOD JUOWAT[IMIGeY yUOS so[9pour 
sg] ‘army uone;ndod ua uoneumMsa,] op sed a] sueq 
“UONeAIOSGO 
onbeyo mod x op smgea sop dsanoo]Joo s0uUENTUT,] 
JoumMsaI Mod IIIA] Ip Sayfa say JoyNoyeo yuaanad saysATeue 
sg] ‘oaneoydxo ajqeueA oun,p snd syoduios uorssaxdo1 
eB] puend ‘sjomMIqeyul snpiso1 sop ke sogld0sse jUsUATeS9 
JUOS S]I,S JJapou np juswa\snfe | suep sjusNUI oNa,p soqqu 
-daosns ‘uanbgsuos red 4a saaneoydxa sojqeuea soy mod 
sovuRLioge sIMa]eA Sop juepassod seo say JaJoda mod sagsTWN 
yUSUIWANbaY SopoyJoU sap jUWOS ‘IDIAZ] op sjayo sojodde 
‘xneuoseIp s}UDWaTa Sas jo neadeyo somneu ey] ‘sogysnfe 
smgfeA sop ‘mo} Ms] e 4a sanourered sap suoneWsa sap 
SINI[VA Sop UOHLUTLUD\Op kB] INs 199IJaxa Jnad souuop ap yurod 
onbeyo onb souanyut,| JomMsoul op d]1WN JWSANOs 3s< [I ‘sanbis 
-SE]D OIIeDUT] UOISSOIZDI op sonsouseIp sureyiad sueq 


uoygonpoyuy “Tt 


: $9] SIO 


‘s}Uon [Jul jUOs syutod sjonb Jouru9}9p Inod aB8epuos ap spiod sa] sed sgigpuod soured sarpulowl 
so] onb joinjd soeUIpslo soued SoIPUIOW Sa] jeSI]YN [I,s 1us}qo yreLMod a\sAjeue un,nb syeqns9I sa] Issne suoseduros 
SNON ‘sopueis juos splod sap no soaneordxa sajqeueaA sop sinayea so] onb sosed saneguly sojapour sop yuoulajsnife | 
suep sjuonyul 99 JUsIeuNod Inb sjulod sa] Jaj909p Inod JaIAZ] Op sjajja sap yo neadeyd soujeul ek] op uUoNesINN,| 
ap suey spore juasgid a7] “AA odepuos op spiod say 19 K SaAneor[dxa sajqeLeA so] ‘A ssuodal a[qeuRA ke] JUOS DOUNNUI,p 
S20INos saj ‘ajgnbue.p soguUOp op opie, e s[apow un oajsnfe uo nbsioT ‘sjsnfe sjapow un Ins ajyueyodun sousnyuI 
aun jusoJaxa Inb sjutod ap sadnois sap no sjonplArpul sjutod sap J9}99}9p op JyI9fqo nod jUO uOIsseIB91 Op SONsOUseIP sa] 


guINS IY 


RIVA pawyory yo rT nyzueir 


dsepUos ap spiod sa] red sgigpuod Ad{Ag] ap Sjasja Jo NvadeyD sd1L.ye 


ZL 


onBoyeyeo ne X-100-Z1 .N ‘epeued anbysye}s 
22-14 “dd ‘| .N ‘SE “1OA 
600z uin! ‘ayanbua.p sanbiuyse) 


ando[eywo NB X-[00-ZI ,N ‘epeued onbnysneis 


‘Bela A-Jasuuds : YIOX MON ‘suljdung facing 
paisissy apo (7661) ‘f “URUBEIM 19 “ “UOSsUaME “q-"-D ‘TePUIES 


‘V6C-6LT ‘SS 

“L8L-O8L ‘76 ‘UOUDIIOSSP ‘anbysijojg ap apouoyousajuy anaay ‘asuodsaruou pue 3urjdures 

JDONSIDIS uDdIdaup ay) fo ouAnor “Suljdures osseyd-omy aseyd-om} 0} suoneoydde ujyim uonsajas jo saseyd om) 10} 
Ul JOJLWIYSS UOISSdIBII SY} JO} UOLUISS doURUIeA (L661) Wa JUS UOIEUIISS JO MATA [e1NUd3 VY (186]) “ ‘WOssuaMs 39 “q-"D ‘JepulRsS 


aBejeo sed sunayewjse dane saseud xnap e eHeuuojlijueYyos,| Nod abesiA nesAnou uf : jepuses ja OBAD}Sy OL 


ongojeye9 ne X-|[00-Z] oN ‘epeued onbysneis 


"86-68 
‘ez ‘ajanbua p sanbiuysay ¢, sgisop xnop & UOT[HUeYD9 UN kB aI[9 
-JUSIAUOD aftupyoe! Np spoysut BT (L661) W'C TANS 19 “S'd NOY 


‘SOUSIIDIG {O }OUANOL puvjvazZ Man puv upiDp4jsnp (suep 
ayiesred e aydacoe juowInd0q) ‘suljdures oseyd om) ul UOTeUITSO 
sourureA (900%) “ ‘ezIzeH 19 WNP ‘ory “WW ‘nojsompry 


‘O@-11 ‘pz ‘azanbua .p 
sanbiuyoay ‘soseyd xnop ke daseuUol]yUeYod.| SUep sollel[Ixne 


soguuop sop lojdurq (661) “d-O ‘Tepues 39 “WW ‘nojsonpiy 


“69I-LSI ‘LZ ‘azgnbua p 
sonbiuyoay “3{qnop a8euuoyjnueys9,7 (1002) “WW “nojsonpry 


"SST-EET ‘81 ‘SousUMIS [PIO 
fo jousnor ‘Suydues sseyd omy ul uoNRIqed oJ UONeULOJUI 


Axerrxne JO sased ud} ou (Z00Z) “A-'D ‘Tepes jo “WA ‘oRvaaisq 


‘OST-IPI 
‘17 ‘ajanbuap sanhiuysay ‘omeiixne uoyeuuojUI.p xnedAlU 
sinaisnid ap aduasaid ua sJyeUaye s}UDWIASsoIpoYy ($661) ‘4 uodng 


‘sueg “FA'S'N’| ‘alsojopoyjaypy ap saauanor sap sajap ‘9stje1g9ue3 
adejeo red asuodaiuou ek] op uoys0LI09 eT (Z00Z) ‘O-f ‘aI[IAd 


T8E-9LE “L8 
‘UONDIIOSSY [DIISYDIG uDIIA‘auUp ay fo jouAnor “Burjdures AaAins 


Ul sioyeUIsa uOnRIqey (7661) “A-O ‘Tepuies 39 “O-f ‘ol[lAoqd 


'68-S8 ‘UONeIOOSsy [POSES URDLIOUTY ‘spoyjayy 
yosvasay Aaaung uo uoijosag ays fo ssuipaaz01g “yoeoidde 
poyipow e - suijduies oseyd-om} Jopun JOyeUIsd UOISsoIsOI 
posyeioueds 9y} JO} UOTeUISS sOURLIVA (8661) We ‘UOs|oxy 


aly deisorqig 


‘soseyd xnap e aseuuo]]nueyos .p sueyd sap snos 
OURLIRA Op UONeUISa,p onbissejo spoyjow ke] anb sovoryyo 
snjd soueueA op Inajeusd uN kB nal] Issne suUOp ayooidde 
ayeo ‘opoe.| syUOUT a] oWWIOD ‘anbyewsa}sAs oroTUeUI 
op JULI[NsoI sMajeUINsd say JosATeue mod yuaIAUOD aBeTeO 
red oysoidde J ‘aseyd owierxnep op a8ejeo a] Inod sanne 
SION so] jo aseyd ssotweid op o8eyeo a] nod un ‘syuaraqzIp 
Solel]Ixne Sin9}d0A ayenb suosstuysp snou ‘sanbysiuayoe1eo 
soo saide.q ‘soqusIayjip uoneunoyur.p sonbrysuajoereo 
Sop juo sodA} stony sag ‘saseyd xnop e aseuUO][NUeYD9,p 
suejd xne sagioosse soleiyixne sojqeuea ap sjualayjIp 
sodA} ston 9)sTxo ]I.nb spore juasaid a] suep suossteu0de! 
SNON “SUONBUISS Sap 91DVIJ9,] SP 9] B] ISO aSeTeO a] Inod 
oeTIXne UoNeULOJUI.p asodsip uo ojanbey suep amsoaur 
ey] ‘ued ne yodder sed steiq sues yuoweAneurxoidde sma, 
-BUIT]S9 Sop JogI0 Inod aseyed red oysoidde.] Ins suonuas 
-U0d SNOoU SNON ‘oseyeo Jed uOHeWMsd,p syooidde oun no 
UOISSoISOI Be] Jed UONeUMMSd,p sydo1dde oun arAIns jnad uo 
‘saseyd xnop e odeuuo]]HueYs9,p sueyd say mod uoneun)sa,| 
op osepuos op uryd 3] ms sgpuoj satjoodsiad aun sueq 


uOISsSnISIg. “[] 


Sl 


cr'0 000 ¢ 

970 8r'0 000 € 

8¢°0 970 cr'0 000 

000 | 000 ¢ 000 € lu 
u 


ZO] Nea{qQe} a] SuEp sajuEp 
-uodsaii09 sagajua xne [‘Q] Nva[qe) a] SUP sag.zua sap ONY 
Por neaiqey 


‘901 Jed soorjdyjnw 
sogjuassid sinayeA xne sa[es9 jUOS Sa][991 SIMaTeA SIT : PION 


I€O17 €€ 000 ¢ 

7S'PS8 9 9 6h 7 000 € 

6L'687 I rI'v9€ ST ESI 000 7 

000 I 000 ¢ 000 € lu 
u 


(744) 4 aoueed ap inayeunsa,| nod uoye[nulls ap aoueeE A 
COT nesqey 


‘01 Jed sogtdyjnur 
sogjuasoid sinayeA xne sojesd JUOS Sd|{[991 SINd]eA SOT : BION 


880% ££ 000 ¢ 

69°SS8 9 SO’ 6rr 7 000 € 

[067 I 80°P9E eT EST 000 

000 | 000 ¢ 000 € lu 
u 


(MNP dT) 4 guiquios 
Npisai E IDULLILA Vp INI9JeWISI,| INod UOHE[NUIIS ap aduRLIEA 
TOL neaqe 


‘901 Jed sagijdyjnu 
sogjuasgid sinojeA xne sa[e39 JUOS Sa][991 SINaTeA SaJ : LION 


76566 €1 000 ¢ 

6L'908 I 79°6LI I 000 € 

76 b8P 16°56 78'¢9 000 

000 I 000 ¢ 000 € lu 
u 


(MP dt) “4 sy2uysip 
SNpIse. & IULLILA VP INaJeWISa,| INnod UOHE[NUIIS ap adURLIEA 


Or nesqey 


(Clede7) 4 ap uONRTNUIIS ap SOULLIPA PT 
ap anioul eT ap surowl e ajeda ysa (“"” 42,) “4 op uone|nuns 
ap soueuea ey ‘uonendod ayao mog -(“!'?4%)”4 eB udder 
red (42) “4 op Jnejar o8ejueae., anuour p90] nesyjqey 
x] “‘smnayeuINyso soyne xnop saz mod onb ("’¢4%)*) 
mod sajqiey snjd juowaneynogjseds juos soqyiyo sq 
‘COL 19 TOL ‘LOL xnesjqe) xne sagjuasaid yuos uoNenums 
op sooURLIvA Sag “YY AP SINIVA QOD ODI Sep UONe[NuIIS 
ap SOULIRA kT gINOTeD SUOAR sou ‘aoURLIA ap smoyeuMse 
SION Sop gOVdyjo,|] op omsaw oun MuUa}qoO.p uyYy 


600z uinf ‘ayanbue.p sanbiuyos, 


CP PeKGe A CORED 


ony, CH) 


000 $z8 
“bbe rey 


az ¢ 
ses Fe Ci POD RN HTD Hp 


‘UOTJL[NUIIS B] SUEP SadST[QN UOT[HUeYOS,p SaT{Te} Sane so] 
Inod sajqeiquias s}e][NsaI sap nUa}gO SUOAR SNON ‘6 UOTWIAS 
eI & agquasgud asAyeue,| aindde mb a0 “("'°4*7) "4 ap aqjao 
@ a]qQejquies yoy 3s9 (° “AA ep voKNquysIp e] “onb Issne 
SUO}Z]SUOD SNON “syNOD snd yusUIa|qeIApIsuCD soURTTUOD 
ap sayfeAroyut sep ynposd qr ‘auueXour ua ‘yo ("1"4%x) "4 
anb sovoyye snyjd sa Cena A ‘ouod “agnus.u09 snyjd 
dnooneag 3so (""" 4%)*4 syounsIp snpisal & IMayeuINsa | ap 
uonnginsip ey (4%) 4 yo (4%) “4 mod aqueddey 
uosey ap JuaIayJIp mb suonnamsip sap a[aaer amBy eq 
‘000 T= 4 190007 
= 'u mod 4 op simaea E00 00] Se] nod suonnqusip 
so] aquoUI [OQ] amBy ey] ‘soueLeA ap smayeumse sion 
sop unoeyo mod sagstjear smajeA OOO OO NuaIqo suOAe 
SNON ‘lor sagjuasgid sed yuos au suorssaidxa say ‘aoedsa,p 
ane oseyd onbeyo e sytogds ysa Sq] puenb (9°6) 19 (€'8) 
‘(€-L) ap yUatnosap Inb saanoadsar suotssoidxa soy sstyyN 
SUOAR snou ‘e[390 Mog C0 A soueLIeA ap InayeuMso] 
yo (de) “4 aUIQUIOD NPISaI B DOULLIVA op IMojeuINsa | 
(mPdzy) 4 “s}OUNSIP snpisal e soURLIRA ap IMayeuMsa,] 
g[hojeo SUOAR snoU ‘s}ey[NsaI sao ap UNoeYD Inod saguUOp 
S3] Ins yUeseq snou uq *(s ‘'s) suoy{WUeYDE,p seamed YOO OOI 
gsijval suoAe snou ‘(u ‘'v) uosteulquios onbeys mog 
‘sonbyuopt 
quos ("'°477) 4 yo (PIG) * 1 39 ‘aseyd oun e adeuUO] 
-[WueY99 || e JuaTeAmnbe 4S soseyd xnop e& oseuuo][HUeYN9 | 
‘u =u Ig (00010007) % (0001 ‘000€) ‘(000Z 


‘000 ©) “(OOO T ‘000 ry) “(0007 ‘000 r) ‘(000 € ‘000 r) 
(u ‘'u) sayte} op SUOSTeUTQUIOD SdSIDAIP SOAR TTTPARD 


SUOAR SNON ‘dSTUIdI sues a[duIIs aNojyeg[e oseUUOT[HUeYD 
un ayiusis syq ‘toy ‘'s op yueusaold sgytun u op Syq 


ando[eje9 nev X-[00-Z1 ,N ‘epeued onbysneis 


Anod sagsi[v91 SANI[VA YOO OOT S9P UONNGLYSIG [OT ens 


"DA 


’ ae OSL 000 S78 


000 008 


000 dees bee b ay 


i m= se eae 
i=) 
So 
i=) 
N 


000 
000 9 


000 £ 


iH ceo Ss oO Ff GO 0 


000 8 


000 6 


000 OI 


000 II 


un 3so S ya ‘f ap yueUsAold saytun 'w ap syq unjsoa 's no 
‘(s ‘'s) sqjodai suojy{yueyos,p samed sap 919 suoAe snoN 
"6 UORDeS BT B onbipul 
yso JI owuioo “g = “x op uoseq suoae snou ‘arIey 
30 mod 4a "4 e gjqeireduios "4% Imayeumsa un Imuyep 
ap suo e (° “ id anes Os “ rareduroo sed suoanod 
ou snou suepuada7) Mtn) yo (M427) 4 roredutoo 
mod uonejnuns oun Jamoexe,p uye g = “x MoAR,p 
amessaogu sed js9,u [I,nb 1930 ap syodun yy (nn “XY 
‘S‘T19 6€ “000 S)= (7% "mM *Z “Nn 2 ‘AV)_ mayoaA 9] Jed 
SQUUOp snuUOS xnv}0} sa] Ins aseyeo ed "AMY = "IG 
mayeumse,; mod %™ xneuy sprod soy auruuajap suoae 
snou ‘'s aseyd arated ap uojjyueyos anbeyo mog 
MIP dey sgsueouly] SULIOJ es 39 74°{ OOAR JoTTIeARY 
suoanod snou ‘uonejnuns aya0 mod 4uanbasuos eg “4 K 
yo 4% ap onbnojduiAse souajeambs,; mod suonrpuos 
Sg] JUOJsNes suonvoyiosds sag “*x = %z 0 g = x 
6p — (“Ay « (din 1) — My 99aP i ie tx Mag) = 2 x 
asl]yn suoAe snou ‘aseyd ouwierxneap ap asejeo a] mog 
(8T19 6€ ‘000 S) = ("2X ‘y) nuuod [e}0} a] IMs adeTeO 
zed sgutuuajsp a9 yuo 's 5 y mod *m sptod soa] 4p 
yuowenny “x = "zy (nq) = "x omnenrxne mayooa 
3] as]yN suoae snou ‘aseyd osouaid ap aseyed og] Nog 
‘coz ase = 4 °X =X Jed guuop 4 ap uonendod op ][e}0} 
g] 1S2 DdUdLIEdxa | SUBP UOTeUITSd,p 9]QIO eT *[ BdURLIRA Op 
12 () DUUDAOU ap plepur}s s[eULIOU IO] e] 3S9 (Q)[eWION No 
‘(o)eUUONS ~ 73 daa “QODS ‘ZT = ¥ 73 + “nE 
+ ™n + QO] = 7A uojas jO19}UT.p a]qQeLIRA ke] Op eae S3] 
9919 SUOAB snoU ‘nal] aWeIxnep UY “Q < x mod , a__,x 
_L(@)1] = (*)f sysuep oun e (p)euueD ro] eB] No 
‘“(g)eurmene ~ %n ya (p)eurUeEDnz ~ "n sajuepuodoput 
salloyegye Sa[qeuea sop suonestesl 000 S red QoOS “” ‘Z 
‘1 = y mod (%n‘"n) smayjea saj wnpoid suoae snou 


aBejeo sed sunayewse SeAe seseud xnaep e abeuuojyueYo9,| nod abesia neaanou uN : |epuses je OeAD}Sy vl 


andojeje9 Ne KX-[00-ZI oN ‘epeued onbnsnes 


‘nary Jotuiaid ua : aUeAINS UOSey RP] op sode}a xnop Us SsoyTUN 
0006 = N ep uonendod aun 9919 suoAe snoN ‘(9’6) Jed 
Quuop (x) A seseyd xnap e uorssais9i e] red moayeuso | 
ap Injod eB s]qQRIQUIAS jso JOIWIOp 99 op JuoWoLOdUIOS 
gq anb jo “(¢°g) zed guuop ("1"%%x) "4 guiquioo npisar e 
QOUBLIPA op INayeuNsa | onb aovorya snjd yuourayqesepisuos 
ang yned (¢1) zed guuop ("'°4x)"4 sjounsip snpiszi 
B QOUPLIPA op IMajyeurNsa., onb uonesayje,| Jopyea mod 
uonejnuns aed oun suojuaseid snou ‘uoTdas 37399 sue 


UOHEINUIS “OT 


“QJURAINS UOI]DS 
RB] & OILIONp UOLNUNS ev] ap UaXoU ne 1099 yUoWIONbuTdurla 
suoAndde snoy ‘uondes ajuasaid ey Be tuyap 7% aseyeo 
Jed mayeuysa | mod (¢°g) suIqUIOD NpIsal B QoueUIeA op mo} 
-eusa,] ap oyooid yuowonbueumu ang wesasp * soseyd 
xnop & UOISSdIsaI eT Jed moyeuijss,] mod (9°6) “QoueLeA 
ap Inayeusa,| Couogq s 3 y mod “a= "J yo s By 
Inod ee) = Welg ‘guinsal uq “Va = “tax AG = tle 
aonb oijiusis ead (01'6) Jed guuop 180 T.nb ja “gq ek 
[e839 sa yeIJNSaI a] {ss Ins sajuepuodsauiod "py Ied soorapuod 
soumos say sed 's ms “no sed saaigpuod sauruos 
xnap say] suodse|duial ‘sapeyoooe aqua uorssaidxa.| sueq 


iMod at (OKA) Be YC Uy et 
( !o/ ax"x"p <) 
(ilo fixtiat, cage (1) x 5 4) a} "x ced Aas 


(x! gx) oy (x £4) 
- q ns owt 


v 


II 


q "x - "4 


% 4 My = Wy 


ly «(@) 
suouaygo snou ‘(6°6) red sauuop ‘** & 


op UOTIUIJOp eI JURSTTNN Ud jo (g°6) Op JUeUed Ud ‘sing ' 

: : ‘ 71 Aly Wy py S 

(016) Suousjqo snou *_ (70/7 x"x"p*<) red saiquiowl 
xnep soy jueydnjnuid ug “Nos ba bes LS ae 
i ig SUR Og tie <= No) WMD SZ so 
e Juepuodsa1iod uoNeNbs ayao ap ajuesoduoo vy oes 
Ho /ixy= "7 anbsing (SEM gtx 408: a) gx) "20° 
="4"z"p°% UOJ ke] snos oaIIO9.s Issne “ynad ayeso 
meg Og (*x?z"v8%) = M'zv8 J eysnes von 
-yop red mb ““'g ap suoyed snou “1anuowsp 9] nog 


(O16) ut "lo past tear Al No / Mx tina TO) . 


MEO G _ 
q — 


v 


a 


ULIOJ B] SOs artI0g,s nad "os 4 "x DS 
Cfo / 1x "x "p°<) = *“'g onb saquour ed suosusuru0s 


el 


snou ‘ual]T 90 IIANOosp Mog {4 (176) suep “a sono] 
-OWOY SINI] & SaT]aJ S]I-JUOS “(g°6) SUep UOTIpPpe Jed snuajqo 
79 + Na = "lg snpisor So] JUOUTUIOD ste “(/'6) Sup "2 
aonb soulgur saq juOs (9°6) suep yuemsy ‘a snpisal so] 


(66) i 10) OO" gage XD ey 
| Mo hep ned Bi: ey 
(%412%°X) (Axtzto KK) ag 


anb ajnoogp [I (z"9) op 19‘ (75x x) = *x op yuowiou 
-uommed ne puodsarioa (gE) = Yq no 


(2°6) (Xx £4) 


s 5 y mod SOG = q "“x-“ = 


ly « (2) ee 
( May Zs ea =" 


v 


's > y Inod 


sed sguuop juos ([°9) suep yuamsy inb "a ya 
"lg snptsa. ‘suoryeoyytogds sao snog +("%9 /"%x “0 / Ix 
= Br Gy 19 fig) ty — NN", < oe a baa i ee 

(4 c 

My — EX gone (x “ x) = x opassod 4% moyeumsa 
}00 Gney snjd suUOTUDUT sUOAR,] Snou aUIWIOD ‘a}qereduos 
Jemouod inajeum}sa,| nod snpisai so] suouTUTeX| { SOTNULOJ 
XNop Sd] SUeP SJUdIOJJIP S]I-}UOS snptIsol soq ywIod jonb 
B SIeUI ‘QUIQUIOS NpIsal B IMa}eUINSd,] mod (¢°g) s;NUOF eT 
B J] quIassal (9°6) B[NWOJ eT ‘Ss 3S y mod onb snuajgo ane 
yuaAnod ou “*a aonb tsute *"'a aonb y1ej 9] JUa}a]Jo1 Nb ‘s ms 
SOUIWIOS SaTqnop Sap 3UOs (9°6) Bp sajuesoduO0s xnap sa] 


: Sq 7 1 phere nal Sl Wl x St 
(Gy ak — = 0 108 ee Meee ae 
‘s > y¥ mod ‘no 
: ; S34 s> 
(96) ie "Iq My Mp Iq ‘ae aK + 


SIL SY, 79% 011, 8? S37 Sau 
oo OURO! One kha aan 


Jed QuUOp }S9 JdURLIPA Op IMdyeU}sa INIq ‘dy TUT] 
}So onbusumMuU soUapIoUl,| JUOP 19 UN op sydo1d MayeA ouN 
xnap sno} yuo mb ‘3 ya 73 suopess snou 1g *(p'6) zed 
guuop 73 doAe apuoyuod soy sed ney ou] * NS yo %S sed 
SQUBISOp adRTeD Op sIM9}dKJ Sap JUaNUOD (7G6]) UeUTOIA\ 
}9 UWOSSUdMS ‘[ePUIRS op OULLIVA op mojeulsa,T 
‘suoneorpioads sao 
snos °¢<x mnayeutsa,| 12 wa Inod soueueA op UOTyeUT}sO 
aun suoutuexe Jueujjuey, °"{ e a[qeredui0s mayeumse 
aunuos 42, soae JaTIeARN JuoUaTduns suoAnod snoy 
e Z suep uedap ap sprod sap xtoyo np Jaj9Inbur snou e sed 


600Z un! ‘ayenbua,p sanbiuyoo, 


SUOAR_U snou aonb ayios ap ““¢ a fal ae 93 ap onbryojdurAse 
gougTeainbe | mod saouastxa say yuojsyes suoneorioads 
soy {(770/%x “o/"x) = %z ouuoy ye snos to/"x = *z 
uysp op sjqeuuosier yso pt “x ynjour ‘x onbsing 
Jo"x="Z y “to/"x = "z oanb onbydun sjgpou sed 
dISISSe MoJeUIYSI,| 9p SULIOJ eT ‘anno ug “¢ = “x yo Mx 
= Wy My — Oy conv (tex <x) — x juee ourmos 4% 
suep *x juesstuljop ua nuayqo 7% Se un,p saqjao 
B sajqeiquias sajyatidoid sap MOAe yerAap 3 4 anb sasued 
eB ouuop ejag “x ™mZ aumsa x ap [e}0} 9] IMs INJa9 
ap oyooid ysa nb a8eyeo un yo nuuos 7x ap uonendod ap 
[B10] 9] INs yowxXd aBeIVO a] JUOUUOP ral suep ™™ soyorduit 
spiod sat ‘ouoq “o/"x = "Zz sae ([°Z) red guuop jsa 
™y’ no xm S< quotasp (¢°6) suoje ‘'s ms saaudoidde 
sogigpuod soumuos say sed s mms sowwos xnap so 
suodedurar snou Ig ‘ayoold ysa aja uepuedag -%x "mn S< 
no xy’ onb aya) ‘aseyd owarxnap op aSeeo 
ap uonenbs oun.p dIqUIDWI SWIATXNep Np a|jamiqey sUuLIOJ 
ey sed eu aja “x2 nuuoom **x ap [e}0} np sterq sues 
yuowsAneurxoidde uoyeujsa oun wos (¢%6) aonb uarg 


-( 41 Ne Ale gate 
(*!0/ “x a ce 
aa Ne Het > 
_ POPs AD r) 
I if 
Herta S shew Asda ® 
| ream Gare BAA levy lala 


muayqo mod (76) 39 (16) 
neaanou op juesiyyn ua ‘x sed (¢'6) suep * suodeydural 
snow opinsuy xO, Vxins< quojsnes “34% = 
spiod say ‘ouoq (¢'°6) op SIquIOWI sUIaTxnep sUIIOD 

"xX ouuop "x = "A oaae (76) 32 (16) ap uoNestpNN,T 
“x red (¢6) suep ‘4 s90e]duia1 red suodusurui0d snou 
‘uonsanb 3399 JoIpme Mog ¢{ ase[eo ap sprlod un suMUIO0D 
% JIalgpisuod snou-suoAnod ‘un ye us A ]I.nb jso jue} 
Is ‘suas Jonb sueq ‘(Z766]) UeUNIIAA 39 UOSsUaMS ‘TepUeS 
suep juouloyI0I[dxa saouous sed yuos ou ™ spiod say] 


AG /7 PMN AY 
10 / x (fo/% x" °C) 


(xt wad SS is eal 
ees nl "o/ re) 
(xin = iota Edis 


red s 3 ¥ mod 9uuop so *8 aBe]ed ap IN9\OvJ 9] NO 
13 %p "py = 3p = ‘mM onb suoanoy snoy ‘jueryt{duns 
ud 19 (¢6) SUep UONNNsQns Jed (7:6) 19 (16) JUeSINpOQUT 
ua soynuapt %™ sptod say sone “MSY = ne dIL99 
suoanod snon ‘(¢6) suep sayorjdut sprod sa] suoulWuy 
-9p ‘uoysanb 9390 e& aipuodsl Mog { parle Jed mayeunsa 
un juUR}9 sWTUIOD gyaIdrayuT ang [I-ynod * “4 Inayeunyse J 


(¢6) 


(r'6) 


anso[eje9 nv X-[(00-ZI .N ‘epeued onbnsyeis 


(€'6) (1g - yO + 


("¢ — 4) 20 OK " 'g ( ty BQ) a my 


Iso te Op 
yueynso1 soseyd xnop ke uorssaiso1 ey Jed majyeumsoa.T 


Bo) eb a 1G | fo /axietn i<) asi 


(76) doAR 
‘gk +6 

19 
toy Ene lsg alee ei pe x0 Xe) kis “lg 

(1'6) JOAB 
legtly = ¢ 


v 


Jed juauiaaydedsa1 saguuop yuOsS 
« NSLIZJUI NIATU»» Ne Ja « INatIadns nesatu» ne soaaysnle 
SUOISSOISaI xnap saj ed 's 5 y mod saympoid sanagid 
sMgyea Say] ‘nuuoout yso "x" ]e}01 a] aonb srpue} ‘nuuoo 
yso x '% ~Je}0} a] anb suosoddns snon “'s 5 y mod 
Ind}99A Np senuuod sinajea sap yuos *x aonb isure “x no 
“(7%x‘"x) = 7x qos yo snuuos yuos uoNefndod ap xne}0} 
Sg] JOP somelIxne sajqeuea op moajooa un "x jos 
IOI SUOQUOUW 3] SNOU SUUIOD 
‘soyooid yUsWIoNbUSUMU JUOS S]I STRUT ‘s}UdIOTJIP yUOWUIIIS9] 
yuos sjamouod smayeujsa soy onb sored uauiajoexa 
sed JUSpIOOUOS 3U dOURLIVA Sp SINayeUIT|Sd xnap sa] “WWouWISU 
-UONIPuOD op jyUdUINsIe.] UOTAS nuUa}qo JUoWaTesa ‘(¢'g) 
SUIQUIOD NPISSI B DOURLIVA OP ANa}eUIISd,| DdAe DOURLILA 
op IMayewsa may suoredurod snoN ‘onbisseyo jUsWIBUUON 
-Ipuod op JUUINSIe | JURAINS US “WepUOodsalIOD soURLIeA Op 
IMayeusd uN doge]a JUO snayne sad “« INSLIJUI NedAIU >> 
ne ayne | 39 « Matiadns nesatu » ne un] ‘saqruodsip same] 
-Ixn® Sd99UUOP Sap ople.] & IIvaUT] UOISSAIZII ap soTIpoul 
xnop op juauiajsnfe.,| puaidwios ;noyed op apoyjow Imoq] 
(7661) UeUNAIAA 19 UOSSUaMS ‘[epuIeS ap saseyd xnap ke 
UOISSaISOI LP] Jed Md}eUINSI | JULUDJUTVUI SUOALIDOp SNON 
‘o]apou Jed a\sIsse yUoUSUUOSsTeI UN Jed nua\qo 
4sa He saseyd xnap e& uorssaIse1 ev] Jed mayeumjsa,, onb 
sipue} ‘aseuuo][nueyoe,p soseyd xnap sap ounoeyo ve oseyeo 
red nuajgo ysa_ 4¢% Imayeummsa., onb sored ‘sonbnuapr sed 
stew “« sayso.d »» uos SINdJVUITso xnop soy “oelrxne uo} 
-CULIOJUI SUIQUI BT INS S9PUOJ JOS S]I,s DUIQy ‘soyUspaodaid 
SUOTIOAS Sa] SUPP aJapIsuod suoAe snou onb ¢%q aseyeo sed 
Ma}eUIsd,.| @ aredu0d as oy qed 191 ties ‘moyewysa 
}99 JUSUIUIOS IIOA dp 2I1IN 3S9 I] (6861) UWOSSUSMS }9 [epURS 
red ayqnd mouayue sponse un ms jueseq as ua 42% 
= X Inod soseyd xnop e uolssoiso1 ev] Jed Imoyeurnso 
un a10qela UO (766) UeUNAIA\ 19 WOSSUdMS ‘[epWeS 


saseyd xnap & UOISS3.1391 
B] Avd ANdj}VUIQSI,| I9Av UOSIBILAdUIOD °6 


abejeo sed sunayewijse dane saseud xnap e aHeuugjjUueYo9,| nod abesiA nesAnou uf : jepuses je OeAD}SA ZA 


ansojeye9 ne KX-[00-ZI1 .N ‘epeued onbysneis 


‘soudoidde 
SNPISal Sop UORUTULIA}9p PI 39 DOULLIA ap SayURsodu0s sop 
yuoutsddojaaap sed sgsuesul] SULIOF eT Op JUOWOSSTTGQEId,| 
red juesusum09 us ‘ayqe|quies aoTueU ap suopsd0id snou 
‘QpoyjoUl v] ap SUOTSUd}xa Sad sURC] “SaxaTdUIOD asepUOS ap 
sued saqne.p e 30 soseyd xnop ek oseuUO][NHURYS9 | B onpus}9 
ang jnod (¢°Z) SJOUNSIP snpIsal B DOURLIVA Op INd}eWIN}S2 | 
pua}-snos nb ayooidde J ‘Q] UoNses ek] eB aLIDap UONRINUIIS 
ey red aysnqyt 3sa (¢°g) Ins (¢°Z) op o8eyuRAL,T “'s puvsds snd 
yUsUIaTTaMyIQeYy UoTTHUeYOS,| Inod snuajgo "a snpIsel so] 
Ins apuoy jsa [I,nb sored ‘(¢°g) sUTQUIOD NPISaI B INd}eUIN)SA | 
anb aovoyya snd sa (¢°/) s}OUNSIP snpIsal B InayeuINSd "J 
'§ INS OUIUIOS 
g]qnop oun,p su} eT snos undeYyo ssuTtIdxa ‘soulia} xnap 
ap SUTUIOS oUN JUIASp jURYNSdI SOURLIBA op INoeUINSa,] 
‘s ms onb sowmjso ang yuoaned ou “a yo “la onbsing 
‘7a ap uonouoy oun yso “(M!’47z) SI8 4S ‘ounzay oueTxnap 
Y] ‘ouUa) Jotuaid oy suep %a + t's = "la ounI0y eT snos 
"3 ya Na JOUIqUIOS B augue snou efaQ “(°'"7%%) 1S ean 
ALLO On ae A=(''"%)A auuoy ke] snos “ls aseyd 
aroruaid ap uoyNueyse,] ms onbisse[o yuuIoUUONIpuos np 
g[nosep (¢°g) aUIqUIOD NpISal B DOURLIVA ap MayeUINsd,'J 
‘s > y mod "a red “a yo 'ssy mod “a red "a yueumso 
uo (¢L) & SUOALUR snou ‘ujuq s3’‘a yo 'sa "a e yen e 
mb 90 ua s ja 's mms agsto1o awWOs oun nb Isute ‘’a ya "a 
B yey e Nb 90 ud Ss IMs oUTWOS a]qnop sun “/'a jo "a eB yreQ 
e nb 90 ua 's mms oUMUOS s]qnop oun : saULIa} SIO yUSTUOD 
SOUBUIRA Pk] Op ojueyNsoI uotIssaidxea,J ~"a 3a "a snpisol 
sop suonouoy onb jue} ua yuouIoredas sajyuesoduios sioy sao 
suoumsa snon ‘(%%a SZ “ay Z)aogz + (“a *SZ) 4 
+ ("2% 8X) 4 = (4%) 4 yuoursddojaagp 2] uedgp ap 
yutod imod e (¢°/) s}ounsIp SNpISol B DOULLIBA Op INa}eUINSa J 
‘yuoraygIp (¢€°g) 39 (EZ) tonb usa yueUDJUIeUT suOUTUUeX| 


) S>) s2 
Wg Wea Np Ny nie ak i 


IL, tela | ea ma " ae 
‘« QUIQUIOD NPISEI » SIJIUSIS (JONpISad pauiquod) mod 
42 INIUZJUI ddIpul_]| NO ‘4yins mb soueweA ap IMayeUINsa,] 
suoua}qo snou ‘souIsa soUlio} xnap sap uonNeUWos 
req (% SZ) “'S4 °q ap mayeumse oumu0s /%9 %g "pn "Ip 
178772 SYR suouajqo snou ja ‘s 3 ¥ mod ({'9) 
red soguuop “gx — ¢ = a snpisai sap suoneumse 
Sa] suosT{yN snou ‘ayuesoduios owiarxnep ey mog 
(2g Mp ra Oy 
op IMoyeulyss sWIWOS TA CA See SIOTE 
suous}qo snoN s 3 ¥ mod ae Magy aa eae x 
~™@ = % +9 = %'g oasqnn ayuesoduioo asotuoid 
ey s 3 y¥ mod onb g]quodsip jso,u nb %4 yuauUaTUOO 
"% ya ‘la aonb sored ‘s ms AUMUOS a]qNOp duN,p SULIOJ | 
ev ounoeyy (7% 4 8%) S84 Sq Y0 (19'S) 4 soyues 
-oduio9 xnap sap sinayeulysa sa] Wuyep ap ajduns 3sa J] 


(€°8) 


acl ata I cal Gh 
x) 


LL 


: (D(x) (2) ¥ 
(7'8) gq? ix — 
(M)(x'4) (mM) ¥ G)\(x' 4) (1)4 Y ae SC 
qx - qx Adee 2 
xe ax) al GYxEa) (Dx 7 ey Gl 
qx — q."'x-~ {4 = 440 


‘SoJURAINS suOISsaIdxa Sa] SUOUa}qQO snou ‘(0 "7) op 
med vy ‘surquioo npises a] sjadde ysa “a + "a = “la no 


(18) (STZ) Bagge ( ote Pe 
("oe") a + 

(Mot {+ Naty YX) 4 = 

(Mote T+ Matty 7) 4 8a + 


[ain ee Hy My a) sis mde 


mua}qo nod 's aseyd asatuaid ap uoyNUeYyD9 | 
Ins suouuonipuos snou ‘“([°¢) op jueueg (¢7L) s0VKOTTJO 
snd mayeunsa | @ suorIoredui0d 9] snou ‘pie} sn[g S > ¥ 
Inod stuyjap juos sinbal snpiso1 say sno} no s0uRLIeA ap 
Inayeurysa un yUaNqo uC ‘'s aseyd orerwosd op uoyHUeYD9 | 
Ins jusuouUOTIpuod sed souRLeA op INajeuljsa un 
ITU9}QO & d}SISUOD ‘IDI SNADI Ud sessed ‘anbissejo aydoidde J 
‘'s 5 y mod sonuajqo ane yuoaned "a suoneumysa soy 
onb juessteuuodel Ud (¢°1) UOISsaIdxa,| B SUOALUIe SNON 


QUIQUIOD NPISZ1  IDULLIBA LE] IP ANIJVUIYS| °g 


‘SQOUDIOYJIP soyueyodurt _p issne a}sTxa 
[I steur ‘oyooidde anou oaae uNUWIUIOD Ud s}IeQ suTe}IaI<. 
apessod (9907) ezIzeY 10 ORY ‘Noj[ZoMpIY suep da}uasaId 
saseyd xnop e& uUOoISsaIsa1 ve] Jed majyeuNsa,| op soURLIeA 
eR] op uoneumss.p spoyjsut eq “*x yo “x ms sapuoy 
asejeo red ayooidde anou op daiayjip [nojed ap spoyjow 
es uepusdad (3661) UOs|exY Jed nuUOdaI 939 B DOULLIPA P| 
ap UOneUT}sd,| nod xnaseURAR a9 JUaANed 's UOTTWURYOS 
puvis 3] mod sanoyeo syeynsor say anb yey a] ‘s yed snyd 
uoyuueyse,| ins ‘"a yo 's uol[WUeYde pues 9] Ins 1UEp 
jury “a ‘sjounsIp juamaulep snpisar say ‘(¢°1) suep ‘anb 
We} I] WWI]Jol (JONpisaa ajoavdas) nod 4s yuepuodsa0d 


INSLUZJUL DOIPUl.| 39 « S}OUNSIP snpissI» suULId} a] 
; IT ALy Iq) ML S207 S34 
(1) fren ye ee Cee 
iy ACR Ne ee 
BLN: OIE A 3 
JLy YL aL, SPIFF 'SP7 Guede 
2 indent) Does: 1 Sottmpbowrs dA 


6002 un! ‘a}gnbue,p senbiuyoo, 


« S}OUNSIP sNpIsal » 
OdAB OURUIRA op INa\eUIT}sa,] Muajqo Imod sayueynso1 
sajuesoduiod sioy sa] suouuOTIppe snou ‘sing s 5 ¥ 
mod "a red "a yo 's 5 y mod “a sed "a amp-e-1sa.9 
‘({°9) red soguuop saayoadsal suoneumysa saj red snuuoout 
snpisol soy (771) suep jueuajuTeur suoderduior snon 


(eaten <0) aie 

[Ca] - | (%°X) | = 
WeacaedaGilin WeR GIGLI): axKNneh Sls 
ah oes tel 0 asaen alge OE: 


(72% (7%p, 14q) rtaig ees ls 


("%a%a Tea Ca ‘sis 89 


SUOAR 
snou ‘aulgrxneap e] mod ‘ajdwioxe reg ‘sayuesoduios stoy 
Sa] INod s]qQe[qQUIas jsa steIq ap ddUAaSqe,| Op dAnaid ey 


Gey i) 
Wz My Mp Iq ar xe Z 
j } S2IRP7 S21 
<I@z Wo Aq a ack 
Ips) Ig5 
<dl5 1 111g ni "& 


qed ‘uawiaansadsal 
‘saguuop jusTelas sayuesodur0s s1oy sao ap sieiq sues suo 
-eulse sap ‘*a ya “a snpisol sa] SUOISSIeUUOD snoU I¢ 


(% te 2p i )aog 7+ 
("o%6'Z)a + ("9% a = 


(ZL) 


(TL) 


(PE) 


SULIO} 
PR] snos saddojaagap ana ynad ai[q ‘(OI y) red stuyap yuos *%a 
¥2 "a no “(][°s) sed aauuop ysa "47 ap aoueLRA eT 


S}IUYSIP SNPISZA 9A IDUBLIVA BP ANIJVUIYS| * 


"SQYUBAINS SUOT}DOS XNOpP xne s}UdIIJ 
-JIp s0ueuRA op sInayeulTsd xnop Joddojoasp jueusquTeU 
suo][e snou ‘souUONUSUISNS s}e}[NSoI So_ 19 UONRIOU 3199 
yuesI]yN u_q ‘soayisod yuos sIpio swWeTxnap ap jo Jomoid 
Op UOIsNUI.p sazt]Iqeqoid say sayno} onb suosoddns snon 
‘ls woy[WueYyos,| Ins ssuuoNIpuos yuos 4% yo /*y “toy 


ango[E}eo ne K-[00-ZI .N ‘epeueo onbysnejs 


My — In'y = MG 
— WpM"p = MQ 


ADM = Mp My jp = My IMNy 7 = Mp 


An _ Il pl aialy <I1ly 


:('s|s39 2% y)Ig =""u yo ('s 3 7% ¥)Id 
=/"4 oIplo swigIxndp ap UOIsNoUul.p soytTIqeqoid 
Sop SUONOUOF Sd] SUTWIOD salUIJap ‘snossap-10 saguUOp 
sgyquenb sop uloseq juouaye39 suoAe snou 4uaAIns nb 
SUOT]INAS So] SUR “[ UONDEs E] e SIUap yuos 7H "'y = *D 
ye Maye) Mou uiy) =o" waBepuos* op spiod'sa77 

‘Ss > ¥ Iod yuswonbiun sgynoyeo 
JUOS SOUINSS snpIsal sa] sno} nO (g UONOVS eT B) DOURLIPA 
R] Op UOHeUIsSa,p onbissejo aspoyjsul e] UOTas NUD}qo ImyJI0 
aonb ("’¢¢7) 4 ap sovoyjo snd mayeurjsa un (7 uoToas 
RI be) amnysuos ap yowued snou s 3 y Inod onb jojnjd 
's > y mod safqiuodsip yuatos *'a suoneumsa say onb yey 
XJ] Ss 3 ¥ mod onb a]qQruodsip jso,u *4 onb sored ‘s SB y 
mod onb sagjnojeo ang yueaned ou *a suoneumsa say anb 
stew ‘'s 5 y mod sonuuood saynoy quos x yo “tx “4x 
onb sored ‘“'s 5 y mod sanuajqo ong yuoanod a 
suOT}eUIsa say onb yso JousI{Nos e yuReyOduT yuIod uy 
fies Pg BoA mod X*' Og uoneumMss uos ed suod 
-e]durar 3] snou anb aos ap ‘nuuoout arooua ysa “OG 
‘uoissaidxa ayoo sueq =“ X Og ty Mg tera ae 
Ng Mp's%) =O eg suuop inb a0 ‘'s ims ssotideadde 
soaigpuod soumwios say] red sagor|duroy yuos /) Ins sowUOs 
sa] ‘“laouautut09 mod : ‘*' “Pg e Joaue mod (7) gq 
suep sinbar jos syuowoxjdurer xneq “(QI’p) suep 
(mx: gry ze ("x %z9%) = (ix: OM ap uoneu 
q una) a] “9 ap uoNrUysp Be] sued 


ly « (m) 
HH Ss 
AMX EA) (MIE Ye My 'S Ae Wate, CS: 
ars tz Tec (x Baio 


(79) ie) 


WEA (MVEA COKE) _ 
OWA: weal ea aot A Wise 


PERE IOMEE HEE: 17 %p sik) bans Ig 
I- 


no 
(1'9) Pe ae GS 
aed Or = Viaactl x a FP aes 

aaa ee wort} leg a 
Is 5 ¥ 10] eat ee 


Oe gqetx + YD) gg Oty ar sL) 


Vv 


abejeo sed sinayewnse dane saseyd xnap e aHeuuogy|!}uUeYyo9,| nod abesiA neaAnou uf : jJepules je OBA}Sy OL 


andoje}e9 Ne X-100-Z1 .N ‘epeued onbysneis 


jUOS ‘(Q|‘p) op ade] B SAQULIOJ ‘SUOTRUIT|SA sag **a 
19 a suoneumse so] red sgovjduiel aj9 JUSAIOP ST] “sTNoTBO 
ang sed juaanad ou sji steur ‘7 3 ¥ 3no} mod stuljap jUOs 
13 49 "la snpisal sa] ‘s}UdIgIJIp syUsUINSIe XNep op JURAIOS 
SNOU Ud g 3a / SUOTIIOIS XNB SUOSTR] J] SNON ‘(1 '¢) red sguuOp 
(“’dcy) 4 goueueA Pe] JaUINsa,p 3so Jnoofgo anON 


JQOULLIBA BE] OP UOHLUIYSI,| op SoTTBUTWITIAg °9 


“SUOT]RALIOp xNap sa] suep JUeLOduAI a[Q1 UN juno! (QP) 
red sguuop *a jo "a snpisei say ‘aovoyjo snjd soueueA 
ap Imoajeumsa un yinpoid ayja,nb sored ‘'s aseyd orotutoid 
ap uo]]HURYE | Ins }USWIOUUONIpUOS op onbisse[D spoujoW 
pe] Jed anuajqo *g uONdes Fe] B 9ILIDap aT[I0 onb ayuessoi9}UT 
snyd ja a]JaANou 3sd / UOT}IS FB] & ILID9P d][9_Q “o]UBSSOI9}UI 
so uosteredwiod me] jo SaIAIns ang ydAned sajyouTsIp 
sayooidde xnaq “4° mod soueliea ap mmayeursa un Jou 
-1uLiajap nod yredap ap jutod SUIWUOD (]°¢) SUOST]IIN SNON 


(1's) ("eZ + te Zs = CPs 


‘quoq ‘aoueLeA ek] & sed anqiguOd au ja yUR\sUOD ysa ‘79 NTZ 
‘QuLI9} Jatueid uos ‘(Q[‘p) red stuljap snpises say 98 (6'p) 
red sguuop “'’¢%{ sgsueoul] SULIOJ eT op souRLIRA PY Jed 
said ap sgurrxoidde so 1 'mSZ = °4% op ooueuRA FY 

‘{ Inod steiq sues juouoAneuxodde 
yso_ "4% ~anb yinsua,s |] ‘ayjnu onbsaid jsa saute} sao 
ap unoeyo ap souesadsa,| ‘p uoNsas e] & QuUUONUOUT suOAR,] 
snou aummog (Og — Og) (x — x) 2 (CM |g 
— OMe) (Oy — Ix) mougyut espio,p souna; xnap 
sop oumuos ev] sa nb ‘“y ap souviodsa,| @ jeda yso 74% 
ap steiq af onb oyos ap ‘y + “4% = "4% “(pp) ap 
NYOA Uq “x Inod sterq sues ysa “"4%_ onb anuout eyaD 


t= soa = (% + "a + 502) Re = Cae) 


nop 
Guejsuoo sua) un yso 99% yo “NaNZ= (Nally 8) Sq 
NZ =(Na""y StZ) °F = (aps) Sy ey = (“ap S%)q 
SUOAR SNON “'s aseyd aroruaid ap UOTTHUBYOY,| Ins yUoUTOU 
-UONIPUOS op sJoMIqey spoyjou ey Jed (6p) Jed souUOp 
SOSUVOUT] SULIOJ eT ap soueIodsa,] suOTNOTeO snou “IanNUOUI 
3] Inog “4% = x mod steiq sues juoweaneuxoidde jsa 
sere ie ‘{ seseyd xnap e o8eyeo red moyeumsa,J 


’d7x saseyd xnap & ase¥9 
Jed 1Nd}VUISI,| IP IUBLIBA 39 SIBIG. °S 


Ade AG 6¥ ly Ay S al I Yay S: Je SCE) 
wy a a 7k ND Xs fe ea) a 

‘A UI 
wo gtx ="'9 juesyyN ud snpise1 say suoumse 


v 


snou }9 ‘(/ UOTOaS FT B ILIONp) DOULLIBA op InayeUI|so OUIQUI 
3] SUOSITWN sNON ‘souRLIRA eI Op UOTeUIISa,] B JAI} 9OUAI 
-QIJIp s[Nes eT ‘saTqe[quias sasueaUT] oULIOJ oUN 39 sq}9Ud 
-o1d sap yuo smayeunjsa,p sodnoi3 xnap sao mod snpisei 
sa] onb sored ‘aseyd oularxnop op ase[ed op souasqer.| Ud 
sjuepuodsa09 $}ej[NsaI Sa] Imu9}qo suoANod sno ‘eoueLIeA 
B] op uoNeUN}sd,p aspoyjeur a][aAnou oun suosodoid snou 
yo "47 saseyd xnop e aseyeo Jed moyeumysa,] op soUeLIeA 
B] 19 SIeIq 9] sUOUTUTexa snou 4uaAIns Inb suonsas xny 
‘2.4 mod “4 =o 70 9 ="31 hres 2 
yuowjduns yuos "a yo "a ‘a mod sayuepuodsaxos 
suonmyap say ‘tuyep sed ysa.u °*'%g onb ays gag 
‘@ ="'x Issne suoAe snou puenb moyeumsa 3990 ap JoIynoy 
-Ied seo UN jsd UOISUd}Xx9,p Ma}oe} B]qnop & InayeUuMso, J 
y= MZ = ("a + Mo + %) 9F nop ‘y anbeyo 
mod “4 = "a+ "a+ > Sueaeredne summi0d (OI'p) 
suep (*'"'™®g onb ofr owtgut gf onof “*'g onb suojon 
‘*“Oq@ suoumiddns snou anb 39 ¢ = ‘x onb suosod 
snou Is (Q[‘y) red sguuop xn9a0 & JUITQUIASSaI SNPISaI sad 


(bI'p) or ne 
a thy = 41, 
Seaaarl Ny = 3 


Nn? 7 
Iod sayueatns suontuyep say yueke “a yo "a ‘9 snprsal 
Sd] ‘S[ANPIsaI sould} sion op (6p) sWUIOS aun dsUTUIOD 
DQSLIBOUT] SULIOJ 9}}90 aILIO9 yUoWIaTe3a suoAnod snoN 


(ctp) Ap ued en )a'x * Cr Oe ly = ee 


mayeusa | 3p 

J9SUBDUT] SULIOJ BY IsuTe SUOUA}GO SNON ‘so10UBI ang JWoAned 
('x £4) lo('x $4) I I 

19 a]qrey snd oxpso,p yuos (°° ”'g— ) UCX— "Xie 

(ANZ SZ U4 7p Sito) Ox —'yx) ‘spueid 

USUUTeSTJNS jyUOS suOT]HUeYDS sa] pueNd) ‘a}Uepgooid 

uonoes ey & stjap wos “*'g yo “CX sauna) seq 


i 1¢ Mg Up exé a 1¢z1p ¢) 


) iS 
wk Vy Mz Mp Cx m ly) + 


(ZI'b) 


OS Hae a Xe 
MD + Cx Oarly £ Cre ly = arr 
sULIOJ 


R] SNOS JURADO] Ud yUaT}Go,s *D™n = ™M oaae saseyd 
XNop & MIeUll}sa,] Op s9sUIegUT] SULIOJ eT ‘aseyd ouerxnap 
Op IdRI[ed J] JULINDEXd Ud sNnUa}qoO xnIo eB JUSWIAAeIedWIOD 


6002 uinf ‘ayanbua.p sanbiuyoo) 


ssoRvOJoul xnop sno} jUoWsTeloues yUOS sIMajeUINsa 
S30 “Uepusdsd ‘sdROI]Ja SUIOW jUdUaT]amMIQey so mb 
JUSISYJIP INsjeumse un ympoid swatxnap eT “4 *v “ZV uols 
-U9}X9,P INajoe} s[qnop ke IMayeuNsd,] suUOp UoNdo aroTW 
-ad ey “on™ = ™m jos “py "py = ty = mM jos OS 
xneuy splod say uswwenbgsuog “» = %m suoae snou 
‘(@ = *x) aseyd owarxnep op aseyeo op souasqe,] ug 


(¢ = 


aseyd sulIIXNap ap Ide[eD SURS SINIJVWISY Zp 


“INdJeUIsd 
399 nod sgsi[Hn ang yUoWTe3a ynod / uOTIas eT] e sasodoid 
QOULLIRA P] SP UONLUWIT}Sa.p spoyjou ey yo “4%q Imod augur 
RI 1S0 askjeue,] Wuepuaday 42x ap sayaudoad say ms suoy 
-U99U09 SnouU snou ‘UdAINS mb SuOI}Os Soy Suep ‘doedsa.p 
gyneJ “sIviq sues yuouaAneutxoidde yuos “?4 yo Pe 
anb ¢ uoloas e] & JoAnold ap jouuiad snou UATIppe mo) 
ea heer (FO. 99 4. 1°a) ie Nop. “y “aribeqo: mod 
MA = "a+ "a+ a ‘seo xnap saj suep ‘anb suojoN 


. (D\(x 6a (Mx 54) (Mm) 
(II rm) (D)(x qx _ (Mx q' x we 
Og Oty Deg ee KS 
(x £4) ('x $x) ('x £4) (xs “gx) vat 
ms a as q+ Q)ax= 


OS ih ey le " QE Bgl atx ityvAly 


‘ ‘ G « (*) 
(HOGS Pg — Cg oe (x: ea ye xen 


OME Ay GAS — 05 


> > ¥ Inod sqjueams 
SUONIUSP sa] JO snpiss1 so] “(g’p) Jed guuop "SG 
Inod ‘augur aq ne 5 yuoutatdums ysa “a aonb suojoN 


‘ (D(x 54) (0) ¥ 
(OI'r) qx - 
OG OFF, fri OE Oe Oty ninigpcs. GTS 
Gxt aq We tly st) 
(M)(x 54) (mM) ¥ (@) Ca) (2) 1 el 
bale Sa Ge pine 


(1x | gx) Na Dee Ory — 0, 


gq Ux + 


‘dn iH 
Iod jins ]I SWS sarUljap JOS saT[onprisel sayquenb ston 
sa] ‘(Sp) red spuuop "x agsuegui] SULOJ Be] Mog 

: Med Cy EOF -STHOIPI 
-s9 xnop sop onbyojduiAse soueueA vy] 49 sTeIq 9] Joipms 
Inod s]qeuaauos yedsp op julod un (6'7) ap ey efaD 


angojeieo ne X-100-Z1 .N ‘epeued anbnsneis 


(67) "a" =X + Mg Mp Ze 4+ 105 ee Ml dZy 


Ya" °< a[qnop uoIsua}xa,p sua} un ja “an “Z aseyd 
sated op uotsua}xa,p sua) un ‘9 yUR\sSUOD sULID} 
UN : SOULId} SION op suTUIOS eB] sUTUIOD “I ¢ 90 Naas a 
rquasgider suoanod snou ‘*a ja "a ‘a snpisai soy 
mod sagudoidde suontuyep sap yueuuaAoyy “(8"p) 19 (S'p) 
qed yUsWIdATOOdsai Sa9UUOP SdgSUB9UT] SOULIOJ sap SUOAIOS 
snou snou ‘““¢ 4 yo “4% ap smoaudosd say Jorpmy9 og 

OG OS Pd ‘kK ap anbyojdurcse ousyeamnbe | 
mod saouasixe soy yeysnes Bag “x yo “x soqueysal 
So[qeLIeA Sep 1n9j90A 91 389 **x no (**x “x)= *x = 47 
10 "x = "z oaae x = “x onb quaanos suojeisuos snou 
‘anbueid ug ((°/%x “x ty) = 4x ‘Suep jred onbjenb 
sasnjoul juatos “x suep sastiduroo sayqeuea say saynoy anb 
uonIpuos e onbyojdurAse yey[nsa1 suIgUI 3] SUOUIa}qO snou 
‘yuepusda, ‘snuuos yuos 'x op xnvj0} say anb sored ‘*x ap 
1x ayuesoduios ey suep 7x amyout,p onbrZo] 389 JI ‘souep 
-IA9 93n0} oq] “*x suep ped onbjanb “x suomypou snou 
anb juowlatnas yarnbor Zp yeVNSOI a] onb sanoy as [J 
‘syuateAInbs yuowionbyojdusAse sinayeurysa sap 

Pd°X “SaUIQUI S2] JUOS SoIeaUT] SOULIOJ sing] 
= "de anb [-p yeynsay np ajnoogp [I 
ae 199 = "yz onb 


WUOSes eet JO 
SUDSINgt ok = 
‘quoq 0 = eOP”g Ng si 
yinsua.s qt {(%*z ‘t'z) =z aonb suosoddns snou enb ja 9 
="y"2 9% onbsing “(7412 9%), (1x1z 9) 1x 


(a2 eS ale fui kaon <i = (e4) qa’ g- 


SUOAR SNON 
‘saazioads suomipuos saj snos 9 = Og?* Vg — OF Og 
anb juenuoU ud ["p IeYNSe1 np s[nossp sAnaid ey 


" = "y no 


('x‘4) 
qd 


(xe) (Ix? x) 


aanadd 


“sjuayeAmnbe yuauanbyo}duAse 
pues EP yon SE ye RES SPP ESOT 
(zz) = "Zz ye (MX TX) = TX IS | Tp soynsey 


‘JURAINS Je}]NSAI J] JULI} 
-ureur suoAnoid ja suodsuous snon ‘sayuejsor so[qeueA sop 
asoduios ysa **x mayoaa 9] no ‘(7x “x) = 7x ouNoy ey] 
ap 3so *x ‘siopy “x ynjout *x anb arruadis mb 29 “(ay1oquia 
aseuuo][MUeYoS | 99Ae aIpUOyUOD sed dU ke) dILOQUID aBeTeO 
NP SCO J] SUOIOPISUOZ) “SUOTIPUOD SaUTe}IIO SNOs JUOS J] SOTI[2 
‘suepuadas ‘saurgut say sed yuos ou °¢% yo “¢% op sagsu 
-POUT] SOUIOJ Say ‘TeIguad ua.nb onuoul Uv yeyNso a4 | 
‘Tp yeynsar np sanaid 
eB] aja[duroo ead ‘yey[NseI a] Suep gOUOUE }sd ]I SWIWIOS 
EOGt De SCD pea ey ee Me Oy HL Sy 
anb suojejsuood snou ‘(g"~) 19 (S‘p) suoreduro0d snou IS 


abejeo sed suna}ewijse sane seseud xnap e aHeuuoy|}ueYyos,| nod abesin neaAnou uf : |epues je OeAD}Sy g 


anZojeje9 ne X-[00-ZI .N ‘epeueD enbysnes 
< LA ly ¢ ) (| 7.) 
ae A) CX XS 


BepcLey + 


(EO, 4 HAY (Pty) Ny eee 


(3p) 


per Ne I OBO A Oye 2 _ un gZ 
(Se Melzer qq tin ywng A 


Md? ap 
OgSLIBOUT] SULIOJ BI B USUI (9'p) SUBP UONNNSQns Jed OULD} 
30 ap uoHONponu,T ‘amesuly uoneunxoidde | yumoy mb 
99 (Og lq _— CHO) (ly — ty) suns) sormaid 9] 
anb a]qrey snyd orpio, p qos (p) ap saute) sxaTWap ston s9] 
anb iuaynos suoAnod snou ‘Jueaviedne sumwi0g “xz °% 


A Wide RAR) ee (“mod steiq sues juoWOAneU 
5 Sarr Sap A WME SEE LL ST RE AEN GP BH BERG) 
(x54) 


=a ages || ayuepuodsa1i09 uonyejndod ap aynuenb es mod 
sieiq sues yuowoAneutxoidde yso SCG IQ «RO 

anuuoou aiquenb ev ap uonoucy oun yso (%' "Me 
anb daIgTUeU SUIQUI eT Op UONOMNSUOS op aad 99 muryap 
SUOANOA SNON “ddSLIR9UT] SULO}J B] JOUTULID}Op op Inq 3] suUBp 
yUsWITOI]du stuIap iso aq “© > ¥ mod onb sonuuos 


yuos ou *€ ap simaea sat onb sored ‘oseyd aiorwmioid 
Ke 76) 


ap uoy{nueyse,| soide.p sginojeo ana sed ynod ou d 
anbijs13e}s ar AN” My ae oxo 8%) = KX oa 


TEM CAP ikl) AR ob. Ol / 2 Z) = ee Yq: aueains u0dsey 


B] ap sarap jos sayyuenb soy ‘uorssaidxa aya0 sueq 


(Lv) ( HX) M19 Vey Ke a 195% Ip) 
) Is 
zl Vy Mz Mp V.Cx ct ae 


PC BA og BE Se eokdg 
(Sieg = Peer an) CS Ee. ly) + 
(ae Rai oa) (xia sg = 


Ce TEDL (x"2" uz) ('x = ly) 
= / d & 


‘JINS [I SWIUIOS dI1IN9 Ss ynad sua} suaLeNb 
Y] ‘sopueis jUOWUUESTYNs jUOS UOT]HURYDE,pP saT]Ie} soy 
puenb (9'p) ap suu9} sUIgINbuTID 3] IaquIo) Jasste] suoAnod 
snou Yuonbgsuod Ieg “IMOLIZJUT sIPIO.p souua} sayne,p 
va (gq - “Og(x - x) MOLPJUI sIPIO,p ULID} 9] 
JUOUWII[CUMIS JIVIDSIAIPNS vI9O ‘ SUID} sUTaINbuTO 9] sUep yUeI 
-n3y x 3 %'g suep suorseydutal sj snou aonb amessaoou 
sed jsa,u Tt “'m aseyd ororutaid ap a8eje9 ap sprod sop suon 
-UO} Sap JUdIOS KX 32 Og anb uslg ‘(p'p) suep sansoy 
-OUIOY SINd] op JuaIEYIp SOULI9} OWIgINbuTD 39 sWaLYeNb sa] 
“dy mod (pp) uorssoidxa,| suep juemsy xnao aonb sourgul 
sa] Juos “4 “{ ap sauna} syotweid sion say 7x “om SZ 
=x 1 2% or el IZ py sy) = mt no 


v 


("9g — Og) (X= X)+ 

Uo a (1x'2% one HEX es 
| v 

te Qatey, Sia, 


«(a) ae if 
ieee Mo 4 OX) gy P)x) Hy <+ 


(9'p) 


'X)+ 


gs) oe A GME: 4) Oi) OT ae 
iene ad Ghutoase mean ciek enema’ 
Jed 9uuop 
“d?y eg auoul snou epag “om = "p ‘xioyd owlgrxnop 
3] snos (¢'p) UOTISssoidxd,[ SUOISpIsuOD “URUD}UTeY 
(S‘p) 1G att Tape 
ly « (*) oy a Ig 
CEM BM, 4 OO gy) Ny <a fi 
(Gx A gy rt (Nx) MY) AFF _ wrdz 
( d x ae d ix) Ke i A 
cp op 


DOSURDUT] DULIOJ kL] JUDSSTUIJOP SdULId} sJomWoId sion sa] 
. di Tb SLO) Pky x Ds 
Oy ae eee 

: D)) ) Ky ' 
=e Gok) ep aN : sjuapgooid =souuia} say onb 
g]qiej snyjd arps0,p yUOs sjI,.nb sored “(p'p) op SOUWLID} SIOTUIOp 
xnop So] Jaquio} Jassie] suoAnod snou “uowtenbasuod 
‘opueis juouUUesyyns sosoddns juowesa ‘s op onpuaye 
are. eB] s2 uw no (uo = ((Pg= 9a 
— x) _N ‘Q]qQe|QUIdS JUOWIOUUOSIeI UN JUAINS UY ‘opueIs 
yuoumuesyjns sgsoddns ‘'s op onpuaye ayjie) ve] sa 'w no 
feleaidoal tin Oe) Cxiax) 

(,-4)°O = ( a - (DX 'X) -N ‘syed 
-9U93 ZaSSe SUOTIPUOS si snos ° aR), @ nod steiq sues 
quswoAneurxoidde ys “* on anb ya 'x mod sterq sues 
1s9 ON anb suoaes snon ‘'s aseyd aiotuoid ap vOTTHUEYOS, I 
ap SUONIUO} sap xnap sajno} yUOS ssouaIayIp say “(0% ae Deg 

ba 'P)q) ('x—'x) umpoid np ysa mb 90 mog ‘ay[nu 
onbsaid no anu jsa souviodsa., JUOp sadUudIayIp op 
s}inpoid sap jUOS SdULId} SIOIWIap xnap say] ‘WsWIsAtdedsa1 
‘s jo 's ms soumUOs sap jue} OUTWIOD sarUTjap ‘sari0} 
-egje sayquenb sop juos nal np sauna} suleysoid xnap 
say “°¢% ap aoueuea ey] e sed yUanqiyu0d ou jo yUR}SUOD 
yso ouuiay oruraad ay 42 9X (*x7Z 92) = “OG (mod 
steiq sues JUsWOAQeUTxOIdde jsa 39) SI9A ay]IQeqoid ua 

2 (xi 4) 

asioauoad %4%z"'p SY Uix'Z"0 °%) = x q sua} a7 

(r'r) (Pg — G(X - x) + 

I © (*) l « (4) 
Ce We = «x bees) ix rt ly) + 

ae ink 


ly « (¥) “Xx 1 
(a#% ax), + One Neg Gtx) Ty < + 


(x! gx) yt (x! 4) OE) 2 sae es 
( ee ee Gi Piccame Sas 


6002 uinf ‘ayanbua.p sanbiuyoe] 


Jed guuop °¢°x suouajgo snou ‘pn 7p 
sl Due. ONY uaulararuarg ‘p mod satqissod 
xIOY9 xnap S2] snos (¢'p) UOIssaidx9,]| SUOISpISUOD SNON, 


aAnadd 
“ 6a lx yg 
Oe ae Pee AEX’ Sex) 

ae ullDd i = AGEs uonenb9 | red Soor[al 

yuos “4¢7 yo °¢4% ap sagstresul] SOULIOJ SOT: [ psoyNsay 


‘uedgp op spiod ap x1oyo xnap say nod snuajyqo 
SIMOJLWISO SO] SUI UONISI PT W1]qQuI9 yINs Inb yeyNsaI aT 


(%O”q __ (x:4) 


(¢p) “(XxX — x) + 


(CEE Mg pry 


5) 
x 9g) Cx = Xe 
ek ee 
4 : l 
AGS: Ms 4 Xx Der P)4x) ty a is 


ly « (*) < 
(a Me tly i eer xix, ol rr 


suuop 7° mod ({‘~) uotssaidxa | suep uonMNSqns 
red (zp) uorssaidxa | ap uononpoyut Tq ‘saywepuadaput soyq 
-BLIRA ap N9}09A suTtUIOD ™'x 49 aJUepuadap a]qeUIRA SUTUIOD 
(OO OgIx ap sostjeiguas uoIssox3a1 ouN IMs gpuoy 
uonendod ey op neoaru ne yuowajsnfe un,p juRyNsoI NpIsol 
mm jugj9 sumo 7% iP), Jajgidiayut_ suoanod aed 
ehe| 
mod tuygp so OF Mat — e), re) 
(MYX IqQ%x Ig xe ("x "29% = (ix De (mod sreiq 
sues jusdionneaicoidde 4S9 30) SIOA Pag eee Ud 39319AU09 
(myx: gx Uy Mp *) (Ux "zp ng) (ix: Pg no 


LOGO Tt eH" 


(Z'p) Cie tind acd) CX be we 


yx! Ex), Hy ae Ox EP) gy ly 
4 


= OO gry Ny 5G 


suouaigo snon “x 9% = xn % aseyd osoruaid 

op ade[eo ap UoNeNbs,| ap yueAIas snou ua 3a “mM mod ({°Z) 

uorssaidxa,| yuRIgsuT us ([7) suep “MIG Mx My 8 

9UIa} a] SUOddojaAap snou ‘ayInsuq ‘audoidde uorssaiaI 

op Jepoul un,p yusWIa\snle | op sed ay[nsa1 aU {I “UOIssaIs9I 

ap npisal un.p me ye *°o 1s owgu ici SUO}ON ‘> ¥ 
mod wysp jo %'q?x = = 7° % ou oT 

7 49 

"7 sa]eJUSUUITLSUT SaTQeLBA dp SIN9}99A Sa] UOT}EJOU ev] SUeP 

sed suonjour,u snou ‘1a1yduns mog ‘sayuepusdap soqeuea 


eBejeo sed sinayeunse ene saseud xnap e eHeuuojljUeYyoE,| nod abesia neaanou up : 


ango[e}ed ne X-|[00-ZI .N ‘epeued onbnsneis 


Sg] No e] ayQuapr 'A jo soyuepuadepur sajqeeA xne s10AUAI 
“A 9ULI9} oJ “UOIssaI91 eI SUep saTqeLIeA so] oyNUOpr 
(‘aA ‘S'A) suo} eT ap soIpuT ua UONRIOU anON « (‘x 
CON RUS Ver B ves justsuuONTyed 2] 4s9 

a tz sy (tx 20°) q 00 


*y 


('y) (“gq - © 9) (x - x) 


lg 
q'x An 4 be 


Bye l ‘ 
(4 )(x ak: Wee yy BOK 4 (x Og ty gece as BG 


W(x Sa) rz (ox $4) 
a°D + 


SULIOJ B] SNOS IMa}eUT}Sa_| SUOALIDD Jo TA MSY = 1% 
“” splod say Joigsut sed suoduawmwo0s snon i 
"SUOTJIPUOD sauTeyaO snos “I” ¢¢y 
=""dy aonb 7p yeqNseI a] SUep SUCANOId snou sTeUr ‘sovual 
-ayjIp jwourare89] yUOS SOUIOJ xnap sag “"!" 4 ‘agsureouT] 
SULIOJ BS suOTNoTed yo “7% sed Imayeum}sa 199 suousisap 
snoN "0", = 1p yedap ap sprod say ms yuepuoy snou 
Use ie suoua}go snou ‘augur aq °"!’¢ vi ‘S9SLIBOUT] OULIOJ 
es suojnojes yo “4% Jed Imayeumsa Jao suoUsIsap snoN 
| OMA Fa ‘uedap op spiod say Ins juepuoy snou 
lone, ‘X SuOUTULIa}9p snoN| “Jo Inod x1oyo xnap sa] xneI9U93 
spiod sao e suomysqns snou ‘anuajqo uorssaidxa. | suep 
‘sing “70 xneiguas yedgp op spiod soy ms yuepuoy snou 
uo 7% ap (enbnojdurtse) sasueouy] auLIO} e] suossI{quia 
snou “‘quowlolatWolg ‘qURAINS kT jSO opoyjVoW IHON 
‘(¢ = *x) aseyd ourerxnap ap aseyeo 9] sed suompdayja.u 
snou puenb assed os nb 99 suouTueXxa snou ‘ojUeAINS 
uonoas-snos ve] YW (¢ # *x) oseyd owiarxnap ap o8eyeo ne 
suop3ooid snou no juessaiajut snd 9] seo 3] SuoUTUTeXa snou 
‘UO}DaS-SNos aJUasaid eT YW “xXIOYD XNap S99 snos In3}eUINSd | 
Op SWUO} ke] Op d9]]Ieap ascjeue oun yuRUa}UTeU suOUasaId 
snoN ‘([°Z) zed auuop aseyd aloruaid ap adejeo ap spiod 
SP so eno 2p Ue ee Mp Mptery = Api (7 
:(7'¢) suep yuem3y %o yedep ap sprod say mod suondo 
XNOp 9}STX9 [I ‘ney snjd suUOyUSUT SUOAR,] SNOU dUTIOZ) 


(@ # *x) 


aseyd JUIdIXNap Ip ISETBD DIAL SINIJVUISY ["p 


suep 


“aBRTVO 99 JURNADATJo UD snud}qo xnod 
anb saovoyja SuIOW yUOWaTToMIGey yUOS Inb smoyeuso 
sane xnap juastnpoid yedap ap sptod say mod x1oyd xnop 
sa] ‘aseyd owiarxnep ap ase[eo op sed suomooyjo,U snou 
Ig ‘onbyevid ua sagarasqo JusuTUaNbey SUOTIPUOD SoUTeYSO 
snos sjuayeAtnbe yuoauanbyo}dusAse jUOS sMoyeut}ss sao aonb 
SUONQUOU SNON “S}USIOJJIP sINayeuINsd sap eB JUSETeIDUNS 
yuausut yedap op sprod say mod sjounsip xloyo xnop 
so] ‘aseyd owWatxnap op a8e]ed 39 SUOMOE]J9 snou pueNd 
‘aseyd oularxnop ap adejeo a] mod uouU no sogsI{Iyn jUOS 


jepues je OeAS}Sy 9 


anZojeyeo ne X-[00-ZI oN ‘epeued anbysnes 


"X somerrxne sojqeura sa] onb yey np juepuedap sjounsip 
sto xNap SUOIEpIsuOd snoN ‘(Z'¢) suep *o yedgp op sprod 
sap uoneorioads ey ap yuapuedap *(%™m *<% = 4% suep 
xneuly sprod say yUSUIUIOD JosATeUR.p }S9 IDI ynoalqo,J 


y.1edgp op spiod so] 
ainod suoydo xnop ap uosteredui0g = *p 


‘Iop1oge Pp UIOS J] SaQne .p & 
suoJasste] snou aonb jafns un ysa *z 3a "Zz mstoyo yUOUTUIOD 
ToAes ap uoysanb ey “*x yo “x ap sax suonouocy sap 
yuos *z ya *'z smayoaa sat anb suosoddns sno °’x onb 
UOISUSUMIp oUIOUI eT e *Z o]TeJUOUINASUI o[qeURA eT “*x op 
UOIISOdWIOS eB] UOTAaS “KX Aap SIeIq sues jUWIOANeUITXOIdde 
no siviq sues Imoajewnsa un jyso "x!o°X =X no 


(Z'€) {¥%z vines "Z Ox —~ x)+ 1} =m 


qed 

Ss > ¥ mod sguuop juos xneuy spiod sag “KX = *x™m SS 

ase[eo op uonenbas,| op ayuTeQUOS eB] snos “ xneuly sprod 
So] suouTuua}9p snou ‘"o yredgp ap sprod say yueyseS 

‘({'Z) sed guuop oseud asotwiaid ap aseyeo 

ap spiod oj iso “mu no’ ““n “n= 4p (7 10° “Mo = 4D 

= 'p ({ : soyjemmeu anne] jo oun] JUaTquias Inb ‘soayeu 

-Ia][@ XNOP SUOIZPISUOS SNON “70 say Inod sfqeuUOsTeI xIOYo 

un.p snjd aystxo [[T “s 3 y mod 7 sed oseyd swierxnep 
op ode[ed oa] nod yredap ap spriod say suousisap snon 

“UOTNTOS suNINe aIOWIpe_U 39 aUaISYOoUT 

ong yWeunod asejeo op uonenbs | ‘uoulg “*x suep sIoj 
gines oun.nb Mmuasims ynod ou arornonred sqeuea ou) 


()4y Ny ES 
lj 
(Ay My ve =X 


ee ar 


g]idurs oiel[Ixne M9}99A 9] SO KX NO ‘K 

= "x" °% so aseyd owiarxnep ap a8ejeo ap uonenbs.J 
‘gseyd owiarxnap op a3ejeo np imped 
B SINIVUINSO SIOAIP op uOoHONpold x] Imod je1gues sped 
UN dIJO Jd XNIp Sa] INS NO sIN9}99A $90 ap UN,]| IMs aseTVo 
9] Jomoayjo,p youuied snou *x ap uontuysp ey suep ‘x 40 
“OX sroy ey e aINpour,p wey oy “”*x 9% mod sterq sues ysa 
aiquiaul suatxnap gf ‘Top "x" SZ = NUS aseyd 
SUISIXNIp op ose[ed op uoNeNbs,] aMejJsyes suOTNOA snou 
safjanbsa] snod satjao juos ‘x suep sastiduios sajqeurea 
soy “Vx 9% mod siveiq sues uowoaneunxoidde jsa 
aiquiaut suratxnep af no “xn SZ = “xn SZ aseyd 
sUIgIxNEp sp sse[eO ap uoNenby,] aNejsyes suo[NOA snou 
sajjonbsa] sod saqjao yuos “x suep sasuduios sajqeurea 
say "x suep sasnyout efap sed quatos au sajja,nb uonIpuos 


(I'¢) 


g 


BOX A XO oX WAQuiesus,| ep sajeo ruued seguuoy 
-oajas yuos “)*x ya “)%x suep sasuiduioo safqeuea so] 
‘ls = y mod sajqruodstp sed 
yUOS OU SINITeA Sma] onb sored aseyd siotusid op aseyeo np 
SONJOXS JUOS SaTqeLIvA sag ‘s yhod snjd oseyd owerxnep op 
uo][HUeYD9, | Inod sayqeArasqo yuOs steur ‘'s aseyd o1gtuIoId 
ap uOTTHURYSS pues uN Mod asnaynod say) jSd SaTQeLIeA So] 
INS S99UUOP ap 9}99]]09 e] NO SUOnENIS sop SuUep SNUUOD }UOS 
uonejndod ap xnej0} say JUOp sarqeueA sop JosI{Qn suoAnod 
snou ‘uonbgsuod Ieg |X op jueuaaoid sajqeuea sop 
suuaiduios ‘%x aonb youued ozpeo onou stew ‘x = “x 
qayiogds suoanod snon “InojeuMs9,| sp soURIRA ek] sIINpsI 
ap ue a[qeiajoid yuowiozjanjiqey yso mb 90 “x suep 
antereddeai yuoanod *'x suep sonuayuo0o sayqelieA Say “XK 
suep snjout ang yned yo nuuos ysa_ “x2 e103 9] aonb ay 
AUBIS BID |X AO 4X eMQuwiosua,| op sofjeo ruued saguuoyn 
-ogjgs quos “)*x moj090A 9] Suep JURINSy sajqeueA so] 
‘aseyd owlorxnap op 
a8vyeo ap spiod sap uoneuruuajop ej suep 39 KX = ‘x’ SY 
aseyd oularxnep op osejeo ap uonenbs | ap yuoulasst]qe}9, | 
suep S}UdI9]JIP soajo1 sap yuonof sayuesoduios sion sad 
"S3][9LI0JN9A SayuRsodulOd sto sop sun.p snjd suep Jamsy 
ynod ou ameljixne sjqeuea sunny “UlO] snjd sayLId9p ‘sayuRs 
-oduios ston puaiduros ()%x XOX) = Tx moaz90A 
SPA Sage op Inojeumss pr ony. Fare 
Jgqnoyeo mod suostjyn sat snoN “Mm (xneuy no) aseyd 
SWIdTxnap op asejeo ap sprod ap sjquiasua un amnpoid mod 
soneyixne sojqeueA op *x mojooA UN suUOST]N snoN 


aseyd aulgixnoap op asryey *¢ 


“pre} snjd yuonteredde nb (7 30 S 
INS) SaT]20 39 ([°Z) suep 's Ms a]]J9d sUTUIOD ‘saTqISIOAUI JUOS 
sooujeul saz onb yuowIanbryeuissAs suosoddns snou ‘ayins ey] 
SueP 19 19] (Z00Z) A[!AeC sueP 19 (OO) [ePUABS 19 ORASISy 
SUP d9}NOSIP jSd [RJUSUIMYSUT m9jd0A UN.p UOHeRsTHN,T 
‘aseyeo op spiod sap ayeisuas snjd uoneoyioods oun 
youuad 39 ‘(766 [) UeUNIIAA 19 UOSSUIMS ‘[epurRS Jed yUWOOp 
ajapour Jed 94sIsse mMoyeUIse,| op s[NWUOJ eT suep “o/"'x 
sovjduior qT “x onb uorlsuowmp oulgul op jeyuowmMnsuI 
maysea un 3so "Zz 19 Mx%"p SY = Xe Cay ne 


; YI ieee > ad a 1 Fi 
(12) ara XUZ"D Pex x)+1} p= "mM 


sULIOJ e] snos 's 5 y¥ 
Iod souuop ywos asejed op spiod sa] ‘uone;nuLIO, anou 
sueq “x 9% = "xm SY aBeyeo op auTeNuod eB snos "!p 
aseyd oigruiaid ap yedop ap sprod say jueyIpoul us saynoyeo 
yuos "m oseyd osoruaid ap a3ejeo ap spiod say] ‘aunone 
aNOA “XX ap safqeuea sauteyso aonb ouuesdwioo ou jI,nb 
90 eB olgtueul op “x ImMJap ap youtad snou [e1sua3 gsodxa | 
‘TOL “3 SUP SonuajUOS sa[qeLeA Sa] sajno} ap anytjsuoo 
wos "x onb amouued op jomeu jos ji.nb uag pres 


600z uln[ ‘ayanbua.p senbiuyoo | 


g[quiasua | ap sooi) samerrxne sajqeuea op x Imayoaa 
um suosijn snou ‘aseyd aioruoid ap aseyeo a] mog 


asvyd a1gtwia41d op asejeD °7 


“SO[QUIDSUD SION S99 op UN] B JUOUUTTedde 
a3e[ed 2] Inod sagsiyNn x sayqeLeA Sa] sayno[, ‘yuepuodsa09 
a[quIssua | Suep xX a[qeUeA onbeyo e juonbiydde.s sauuojoo 
SOINIWIOP SION soz suep soanbiput sajyaudoid say] ‘syisnjoxe 
JUSUTZTJOMINU yWUOS sa[qUIdsSUd SION Sa] “x So[qeUeA op 
siquiou janb syoduir,u Imuayu09 ynad ajquiasua asnbeysD 


sonuuos sonuuos nuuooul & 
sonuuod sonuuooul nuuod \X 
sonuuod sonuuos nuuod es 

s > y¥ anod Is > y A Ans SOTTBIIXNe 

II QUlIvA e| Inod aqetiea QITBIIXNe So[QULIVA 

op selleyiun =—s- & | ap SaaTezuN a[quliea ap 
Single single A BL ep [ejoy «=: ajquiasuq 


saseyd xnap & aseuuoTQUeYd9 | 
Inod saieyixne sajqelivA ap sa[quiesuq 


TT neaqey 
“UONPULIOJUT | DUIBDU0S 
mb 90 ua sonbysugjoeres smmoy suuop yins Inb neaqe} 
oT .X © |X ‘,X Jed sousisap yuos sy] (x sarqeuea 
sogjodde) sarrelixne sajqeueA op sajquiasua no sodA} 
sIoQ Jonsunsip suoAnod snou ‘soseyd xnop e sseuUo][y 
-ueyo9,p suejd so] nog “onelIxne uoHeULIOFUI,| ap PIt]Iq 
-tuodsip e] ap aj3duioo yueUs) ua soovoyya snd smoayewnsa 
sop omnpoid suoanod snoyn %4 9% = x Imod steiq sues 
yso_ "DS UOISUa}Xa,p INd}Ov} B[QnOp e IMayewsa,T 
“sopueis JUDUIUUesTyNs 
quos saseyd owierxnep op jo aiomwoid op suolmueyse 
Sap Sonpusye sayjie} say puenb smayeumnsa sou ap asATeur | 
SUeP INOLIDJUI dIpIO.p SoUMd} soy JoyedE,p jouuod snou 
Inb 99 ‘aseuuo]]yUeYde,p suejd xnoap say 39 UoNe{ndod e| 
INS Sa]qIe} SUOTIPUOD ap asaujOdAY,] SUOWOUID SNON *« ueTd 
ne yodder sed steiq sues Quawaaneutxoldde) » ayr1usis 
« sTeIq sues (yUoWaAeUTxoldde) » uUortssaidxa,, jo ued 
J] Ins sopuoj }Sa SMajewsa sap osAyeue,] ‘apse yUasoid 
gj sueq ‘s > ¥ mod *"%"'p = “bp 4s9 uOIsUd}Xa,p Maj,oRJ 
giqnop e& no 9uIquI0d asepuos ap spiod a7 ‘('s aseyd 
siotuaid ap UOT] WUeYDS | B JUSWA][OUUONIPUODS sgTNdyed JOS 
% 49 %u onb sojaddex as op quatauoo yr { “py yo Sy 
saatjsassns snd sauuoy say aonb yoynjd 7% 3a "x suostyyn 
snou ‘ajdums ajsor uonejou eve] onb mog) ““u/[= "7D 
ysa aseyd oularxnap ap jauuoNIpuod asepuos ap sprod 
a 10° 's.> yeanod. (‘sis 54) 1g = "4r-1so ¥ op sAnisod 
9 ONUUOD a]JaUUONIpUOD UOISNOUT.p ayTIqeqoid ey «'s op 
mued e s aseyd owiarxnap op uoT[HUeYda UN sUOUUONDII9S 
snou ‘'s yueyoes ‘siopy “'s 5 y saqun sayj Mod sagarasqo 
ang-nod yuos safqeuea souteyog “ust = "po so oseyd 
aioruiaid ap asepuos ap spiod gj 10 ('s 3 y¥)Ig = “"u 
soy gun] ap onuUOD aaNtsod uoOIsN{oUI.p syTIqeqoid 


suep aseed 23 


angojeyeo Ne X-100-Z1 oN ‘epeueD onbysneig 


ey ‘'s oseyd ozgnuaid op uoyjyueyo9 un suom snou 
‘IN 7 4 “7 T} = Q soqun.p ory uoNendod ey] ap 
: JURAINS 9] 3So Soseyd xnop ev oseuUOT]HUeYDE.p uLId a7 
‘aseyd owiorxnap op a8eyeo un 
yo oseyd oromuaid ap aseyeo un ‘mu sprtod xne yuessynoqge 
snssa001d 3] suep sadeja xnop Jonsunsip suoanod 
snon "Am °X = 7 op ooueuea ev] dANeOIIUSIS oJOTURUI 
ap JSANOS eIINpal soqueyodun SOTIeT]TXNe SaTqeueA soy IMs 
« SQAQ]9 SN{d xneodAIU xne » ddUdINYOO FR] ‘(aseyd outerxnap 
op uoTHUeYSS) NeoATU Joep ne jyuawoenbrun sarasqo 
yso 4 onbsing ‘nuuoour “x” ye}0} np ‘aseyd osoruaid 
op UOT[NHULYSI,| Sp NedAIU ne o}IeJ ‘sTeIq SUeS (QUOOATIEUI 
-1xoidde) uoneumsa oun (no) 3a UuoNeindod e] ap neoatu 
ne nuuod "x"% ye\o) 939 mY = 4% suep ™m sptod 
Sd] BUA dUSIOYOO oun ye A [I.nb SUO|NOA SNON ‘sogIpnys 
S3[qeueA SoJ no ey IMod sayley suOTeUIT}sSa sap soUeLIeA 
R] DINpsl 39 SaelIXne sINaeA soy Inod saumtjsa no snuuos 
xNBJO} Sd] 99AB JURPIOOUOD splod ap s[quIasua UN 199I0 
: smoyory xnop sed agay0UrI jso ase[eo Jed aysoidde,7 
"6 UOTDES B] & DYLIO9p UOSe] BT ap ‘X dp UOISsaIseI odAy 
ap InayeuTss,| altmysuos Inod sastjyn yuos "x2 nuuos 
]e10} a] aonb isure “{ ap saytpoid smoayea ap sajquiesua xnop 
Sa] “INa}oIpaid sur09 «x majo9A 9] JUOWIATN|S yURSTTWN 
ua 's 5 y Inod yinpoid jsa ayne un ‘{ smajorpsid oumU09 
x yo "x sioy ey e yuesyyn ua 's 5 y mod sayinpoid 
quos uorssoiso1 Jed satpsid 4 op smoea op s[quiosus 
Uf) ‘saeaul] saued soipulour say Jed suorssaiso1 xnop 
Jaisnfe.p Joimeu jso [I ‘UoIssarsa1 red ayooidde | sueq 
“SqUaIIIp sonbysugyoeses sap unseyo yuepessod 
‘omeyixne uonyeuuojyut.p sodXy (xnop onb jojnjd) sion 
JUBSSTRUUODAI Ud SINILUgIUe XNeAR] SI] aJjJo}o |] ‘oseyeo Jed 
ayooidde | ms axe yUOUATeS9 3S9 aoe yUOsSoId a] “xNROATU 
xnap xne s[qmuodsip uonewuoyut,| yuowoonred no jour 
-J[10} Jasl]yN.p sjudiayjip susXAow xIp juonsuysip jo oseyeo 
np juoWMBIe | Ins yUaNUDUOS as (Z700Z) [EPWUIRS 10 ORADISY 
‘soseyd xnap ua aseuuO][HUeYs9,.| Op Sed od] SUP SOURLILA eT] 
ap uoyeUIsa | Inod sseuUOT[HUeYIIEI NP spouJeW kT gIPM 
UO (4661) [2xMS 12 HOY ‘soUaNbesuod op nod je yWRd9,[ 
onbyeid ua nb ajqeqoid jso [I steur ‘sonbyusp! simajeutysa sap 
yUoUaMessaoou sed jusuUOp ou sayooidde xnep soy ‘arerrxne 
UONEULOJUI SWIQUI e] Nog ‘UOTssaIsaI e] Jed oyooidde | uojes 
snus}qo xno & JuRI[NSaI ase[ed Jed smayeujsa sot oreduro0s 
10-9]]9D (S66) JUOdnq sed soseyd xnop ev sseuuoT[NURYS9 | 
e sonbrjdde ay9 & (7661) [ePUIBS 19 AI[LAOC Suep aILIOp aseIed 
np syooidde,J (9907) ezizey 10 oFY ‘nojsonpty.nb sure 
(8661) YOsTxY “(8661) TePUARS 19 No[soNpIH (L661) J9HIS 
(7661) UeUNaI 19 UOSsUaMS ‘TepLaRS “(L86]) UossuaMms 
Jo Jepues suep spoduwia | uoIssaso1 9uN,p JuoWa\snle J 
‘Isure sed yUsuaTeI9U93 So UI_U [I sTeUT ‘SonbryUspr smo} 
-PUII]Sa Sap eB INoge JUsAnad s]I ‘sUOTJIPUOD SOUTeLIOO SNOS 
‘a8e]eO Np Inja0 Ja UOISsoId91 oUN.p yUoWsn{e.] ap mys 
» 4 9F= X voneumsa,| suep onerixne uoyeuuojut p sodAy 


abejeo sed sunayewnjse dere seseud xnap e aHeuuo}|uUeYII,| Nod abesia NeaAnou uM : |EPUeS }8 OBAS}Sy b 


"UW09'S19B01@)|Vpwes ]ILS : JOLUNOD “IMassajold ‘jepweS YUA-|IeD ‘ Bd°9S"ULd]EISM)OBAIISA-IOJOIA : JBLUNOD 
‘OLO VIM ‘epeueg ‘(ourjUQ) eMenO ‘epeueD anbysneig ‘sesudanua sop soidne sajgnbus,p sopoyysul sop UOISIAIQ ‘ORADZISY “YW JOWIA ‘| 


XN9P $90 sp UONLIZOUI | OP INdARj Ud SJUOWUNSIe xnop onje1 
-UI] Be] suep sanoyn uQ “'s op neaaru ne ued ne podde1 
red steiq sues 9umjso ang ynod stew ‘nuuoout so x 2% 
[e101 oy {Ss 3 y¥ anbeyo mod onuuos juonbgsuos sed 4s 
19 's 3 y onbeyo mod agarasqo ysa "x m9}09A Np INgyeA 
eR] ‘UoTTWUeYOS JomUeId np nedaluU ny ‘s S y onbeys mod 
nuuos jusuratesa jso JI Suanbgsuoo sed ‘'s 5 y onbeyo 
mod nuuoo maydaa un ysa "x no ‘nuuos yso x "'% ]e}0} 
3] ‘uoneindod ev] op nesalu ny “‘nedsaAluU Me] op UOTIUOJ UO 
SO9}HTEN JO SONUUODSI JUOS OIIeI]IxNe UONPULIOJUI,p sodmos 
xnop NO 3}xa}U09 I] SUep smayne sImoisnyd red souTUTeX9 
a9 e saseyd xnop e aseuUO][HURYdd snos UOTeUINSS,T 
‘Q 5 's 5 s aonb ap0s ap ‘fQ op mued eam 
's aseyd ozarweid ap uoj[nueyos,| ap uoT]WUeYDe-snos uN 
yso s aseyd owiarxnap op uoT]HUeYD9,| : saseyd xnop ke aseu 
-uo]]HueYse sjodde juswaTjanyiqey ‘aytoquia adAq a] Ans ay10d 


gjome juasaid a] ‘ajloqua uou 42 ajioqua ‘a{qnop oseu 
-uoTHueYoe,p sadAj xnap ep ainosip (1007) NopsomptH 
“(guinsiquie 
aunone ve A.u jis ?"% op sasoiqe ouuoy owmu0s 
uonej}ou ke] SuUOST]yN snou ‘7 D5 PY ‘soyIUN,p s[quUIasuS UN 
iso p Is) = X uonendod ap 4 ]e}0} a] JoWMNsSa_p 3sa 
jnoalqo,J s > ¥ saytun sayz Inod juswronbrun onus}go 3s9 
T€ mayea es ‘a red agusisap isa adAq oaIpnjo o[qeueA sup) 
‘§ UOTTHULYOI | JURAR DAIASQO }9 asTTeal 3S 's UOT[WURYDI,T 
tN SO 4 oT = AQ Jed aguuop IgINUI.p suTy UONe] 
-ndod ev] op sajquiasud-snos sop xnop sno} juos Inb ‘s 49 
's red sousisap ‘sayst{iqeqoid suoyjyueyos xnop op a8eI 3] 
}Sd ouNUTOD onbysua}deIed ke] JUOp oseUUOT]NULYIE,_p sued 
XNB JOUdIIJOI We} ajqnop aspuuoyyunyoa uoissaidxd J 


uoHINpoyUuy “Tt 


“QUIQUIOD NPIS2I B DOULLILA P| OP INd}eUII}SO { SJOUTISIP SNPISoI e OURLILA 


op Inajewnsa ‘yedep op spiod ‘{ saseyd xnap ke uolssais91 ey Jed InojewNso { alelIxne uoleUOjU] 


> S9]D SJOP] 


‘saseyd xnop e o8e[ed Jed Indj}eUINsd UN,p SaT]90 e& sayqe|quuias say9Udo1d sap 
B s[opour Jed gysIsse Inojeuryss,| onb suoMUOU SNON “XNedAIU XNap k& dIIV9UI] UOISsoIB9I Op sJgpowl uN,p JuoWO}snIe | 
syodwos inb onbisseja sjapour Jed agysisse UOISsoISO1 ap spoyoU eT & ase[eo Jed ayoosdde., suoredwiod snou (yp 


‘ spouj}oul 


I[JOANOU 93399 op soadeULAR So] JOPI[VA & Yas Q] UOTIAS e] B BJLOSP UOE[NUIIS sUA, ‘aseYd sJoTWOId op UOT|QURYDI | 
Ins JUSWSUUONIPUOD op sTJoNyIGey onbruyos} ey e yoddes sed a[qeiopisuod UONeIOIWe oUN Jo}UdsaIdaI jnod Inb spoyjur 
9][2anou oun suosodoid snoy ‘saseyd xnop e a8ejeo Jed inoyewysa,| Inod sdueLIeA ve] 9p UONeUIISA,| SUOUTUeXe9I SnoU (¢ 


‘ onbyojdwiAse soueeA sWgUI 


BR] JUO SINDJLUIISS xNOp S9d ‘sUOI}IPUOD SoUTeYID snos ‘JUePUdda. ‘s}UDIOJJIP JUSWIIIBI] SINdJeUIT}Sa sap JUdWIOTeI9UNS 

juosinpoid « sjainjeu» xIoyd xnop sa] ‘adeyed a] Inod yedsp ap spiod ap xioyo sSIdAIp op jajja,] SuouTUexa snou (Z 
‘ odeleo Jed Inojewinso,| op onbyojduiAse soueueA Pe] JUDUTULIO}9p Inb snpisal xne JOALUe Inod uoNRsUR9UT] sp spoyioUl 

aun suosi[yn 32 adej9 onbeys snod soudoidde sosmeljixne sinajo9A sap UOTyIsSodwi09 ev] ap suOINOSIP snoN ‘aseyd owatxnep 

op ode[eo un jo oseyd signed op o3eeo un : sade} xnop ua splod say suep 99139}UI }Sd UOIRULIOJUI, J “UOHeUII}Sa, | 

suep suosI[ijn so] 39 soseyd xnop e surjd soy inod omelixne uoneuuojul.p sad, spues3d sion suoase8ap snou (| 


: soseyd xnap e adeuUo]]yueYyo9 ,p 


sued so] inod o8ejeo np [ei9ugd 9)x0}U09 9] Suep sjueyodu souIeWOp ayenb e uolNquyUCD sun syodde ajome,T7 


‘sonbl0g93e9 No sanuljuOd ag yUaANad sagIpNje salqeueA 


So] 19 Somel[Ixne so[qeuea so] ‘oseyd onbeys ev o3e[eo ap spiod sap [noes 9] Inod saljyUept yUOS oIIeTIxne UOTeULIOJUI,p 
sodA} sIoAIq ‘odeUUOT[HULYDE,| sp aseyd onbeyo ke stud juos sareniqie adeuUOT[HULYII,p sued saq ‘sseyd owiatxnap 
ap UOT[HUeYIg.| SUP SoNUdJUOD sgyIUN soy] Inod jUsWoNbruN agAsasqo }S9 ‘9UIISA 9.19 JOp NuUOOUI UOTye[Ndod ap [e}0} 9] JUOP 
‘agIpNyg s[qeUeA eT ‘Jorwold np uoT[WUeYd9-snos onb jue} Ua 911) JSa aseyd oWIaIXNap op UOT[NUEYE UN sind ‘aruy UOe{ndod 
9un.p Ited e 919 3so oseyd os9twioid op uoj[NUeYs9 UP ‘onbissejo saseyd xnop e adeuUO][NULYDE,p ue]d a] SUOJOpIsSUOD sNON, 


‘ojo Juasgid 9] suep agsodxa saseyd xnop ek aseuUO]]UeYDE .p 


suejd so] inod a[ei9u9s aIso[opoyJoU ev] SUeP Z}9[J01 ISA JHOV[go 39D ‘adeUUOT[NURYOS,p suL{d syUdJOJJIP suep oser[IXne 
UOHVULOJUI,| JUBWIDOVOLJO JosI[NN,p ona usd suondo,p summed opuesis oun ILYJO.p isa yUoWaddojaasp 90 op yueyodut 
Jnoalgo u~ ‘epeueD onbysyei¢g e UONeUISa Pp S9st[e19UIS S[o19I30] ap yuUaUIaddojaagp Np ajinsinod ve] ap yUs;noogp sajuasaid 
xneaey so] ‘soseyd xnop e oseuuo][ueYys9 .p suejd soy snos o8eyvo sed uoNeUIsSa,| Inod oped uN yLOsp spore yuasgid oT 


suINsoy 


| [WPUARS YLIY-[AVD 39 OBAIS|Y “JA 10391 A 


ISv]Vd 1vd SANIJVUIIDSI DAV 
soseyd xnop & ssvuUoT[QUeYDI,| Inod oSesIA NvaANOU UQ 


anBojeyeo ne X-100-ZL .N ‘epeued enbiysne}s 
9L-€ ‘dd ‘| .N ‘SE 10A 
6002 uInf ‘ajanbua.p sanbiuyoe4 


ando[eyeo Ne X-[00-ZI oN ‘epeueg onbnsnrs 


an9]9p Joyo ua Ina}oepey [aque plorey] 


‘sgsodoid spoujsul eb] op soueUIOJIod uu PT SULTJUOS UONe]NUIIS Jed 
opme sup) ‘UoNne[ndod ey] ap nesaru ne aseiqiinbs,| yuessyUeIeS aqnyD Np spoujou! e] op ajdepe oumpUOS]e 
un osodoid majyne,J ‘yoexa aseiqiyinbs un smeutiod mod sayned doy jos ajens anbeyo suep sayjie} 
sg] onbsioy suqiinbs aseuuoy]HUeYs9 | op swatqoid ne onbeye,s JoaneyD ‘apoe UOS sUep 4UOWTeUL] 

‘sanburdura sapnja pp usAOUr ne JaIWap 39 JUaNsnyII 39 9sodoid mayestput | op 
sanbuosy s9j9Udoid sa] yUassI[qe}a SINa\Ne Sa] “‘SddATISQO Sa9UUOP Sap UOTIsOdWIOS ek] Ins doUSprIoUT oun 
no e ssuodai-uou e] anb aanaid ounone 94sTx9_u [I,nb onbijdun ayqrej uonelsosse oun .nbsind ‘uonersosse .p 
souasqe | SP MSW 9UN SUIWIOD Y MaeoIpuUl.| JoJOpIsuOd op y[quigjoid yso [I Ge} U_ ‘somerixne 
Sa]qeUeA Sa] 39 asUOddI Op a[qQeUeA ke] ANUS UONeIOOSse,] amMsow Inb JoweID sp A ne jyutod ureyos un 
e.nbsnf sjuaredde jsa “y mayeorpur gjadde “mayeorput ja9 aonb yUaQUOW s|] ‘“asuodas-uoU ap sTeIq Np 124J9,] 
onjead uo] onbsio] ayyenb ey ap mayeorpul awu0d asuods1 ap xne} say Jooefduiel mod sytAneyUsssidal 
op Inayeorpur un juasodoid sj ‘apms,] e uoNeindod e] no uoyHUeYdd,] 19 aJgnbua oun e ssuods1 
PR] IUD DUETS ev] Op UONENTRAD,] Ins yUOYSUSd as WaYITHIOg 19 UaqqoD “UaynoYSS ‘apoE Ma] sueq 


ougwinu 39 sueq c 


“]IQeUeA op s9dmMos 
So] Ja[ONUOS 49 spusidutOD xnoTUI eB Japre yUoANad sje[NsoI SINY] “SIMOMITAIOUT xne 39 sareneds soddexs 
xne ‘uonelopuod ey e snp sjajjo ua xne}0} s}ayJo sap uoNIsodwiosgp oun Issne juasodoid si] ‘soyeneds 
soddei3 sap 19 UONOaI9S ap saesout sayTTIqeqoid sap jUdIAUOD ]I sUTUIOD 3}duWI09 JUS INb MOMOIAIOIUL,| 
B ONp II]TIqQeLeA ke] 9p UONIUEp oun ‘oxojdwios sseuUOT[NUeYSE ,.p ue[d uN,p 3}x9}U09 a] suep ‘UoTUDT}qGO 
st ‘sing ‘sayeneds soddeid sues sayesa soqyiqeqoid saae oseuUOT[NUeYSE,, Mod MaMOIAIOIUI.| & oNp 
QOURLIVA BT Op anbissejo a[NULIOJ e] Op sJapouw Jed sg\sIsse UONRoTNsnf oun JUSSsTuMO; LTYRT 39 Jo[QeH 
‘sonbuidwia sapm9,p 39 UoNefnurs red sapnjd,p UdAOUI Nev sayUe\STX9 
sapoyjoul xne dIQIWIAp 21320 JUaTedwiod 39 JUaNsnT! 39 sasodoid spoyoW ev] pud}-snos nb sL09y) eT] 
JUDIOQEIA SII ‘UTeUUOSTes op sInayeoIpuT sord say sgIpMja onbisopouosYo sL9s eT op s[eNdeds gyISUSp ke] SUP 
I9}09}9p op Jueyouued onby snes uoNeoyIUSISs ap 3s9} un JUasodold sinojne sa] “sogst[euUOSTessp sopuUOp 
SJ] SULP d]JONPIsoI gyIPeUUOSTIeS P] IP UOT}D9}9p op s}so} say IMs ayod uefoY 39 AoA op sponse, T 
‘QIpeyeul-souemsse, p sed yuepassod ou sonbneisy .p 
suoiiodoid sop sourewiop syyed ms suoneumss sop muajqo mod (STHN) A2¢INS MOIAIOUT UyeoH 
euoneN ke] op yueUDAOId soQUUOP dP JULAIAS 9S UD SOpOUPSU Soz JUANSNI]I S|] ‘syuepuodsou0d suusAOUI 
onbyespenb mox1d,] ap sonbnojduiAse simayeumsa saq yo sonburdwis sustssXeq sinsjewso soy Issne 
yUaUUDTIGO SII ‘sayeumMdo UoneUISd,p SUOT}OV Sap dLIONU} eT Ins s9puo} dydoIdde oun jux}dopy ‘uone[ndod 
ap sourewiop sjjod sap inod suonsodoid sap wouasod e sadAj-smad so] 39 SouUsAOUI sa] I1U3}qQO 
Inod 9si]e19Ud3 9}XTUI SITeSUT] B[OpOW ne osnsoyeue usissAeq onbiyoreso1y sJopoul un yussodod sj “97unN | 
Op NBIAIU Ne Sd]QpoUr sap yURSTTN Us souTeWOp syed ap suep suoTodoid op UONeWMsd,| nod sonburdurs 
19 sonbryorels1y souuatsgXheq sopoyjowW sop judIOgeR]a suosIeg 39 Jjozjey ‘Wreypy ‘eyUIg ‘Wry ‘ysoyH 
"SUOT}NITO Sa] Nod sUONeUIT}Sd Sap s}eI[NSOI Soy 19}91d1UTp 39 JOSI[NN,p JUSTAUOD 
[I JuaUIUOD JoNSNyTI nod oC, | Op 17q,] SUPP 866] US SasT[ed1 XNvIO}DITO SOdEpUOS Sop jURUdAOId soguUUOp 
ap xnof anenb op jueasas as ua jo[duoouT dau aIqnop eB sdUa8uUNUODS op nes{qQe) UN yUOSATeUR ST] “sIOpUT 
Sa] Jo syuepuodal say red s9UUOp UONEUOJUI | op yUopuodsp uoud ke ayI]Iqeqoid op suoNNqusIp soy yUOp 
suaIsgXeq Sajapoul xnap jUajUasoid sane sa] ‘sINd}99]9 $99 ap a}dWI0d JURUD} UD s}eI[NSOI sap UONOIpaid 
eR] op opmnoexa | aois0e ynod uo] onb jo uONda][9 oUN,p [eUTy yeYNSOI 9] IMs aJUeYOdUT sOUDpIOUI 
aun IOAR JNod SId9pUT SINd}d9]9 Sap 3}0A op JUoWIayOdWIOS a] onb gsHUOU yUO sapmM—a saq ‘asuodal-uoOU 
RP] op jUoWIOIeNH op duessalgjur oyooidde oun juajuasoid 1oyD 19 IoOyD “Wed ‘aoe moa, sueq 
“qegUT] UOISsaIs91 
op sopour un.p sonowresed op majd0A un.p jofns ne sasayjodAy say dyUIZA UO] onbsio] TOUayUOg 
ap 19 PIRAA OP Sa]jao onb syeynsor sunaplow ap 3a ‘HooS-orey op ampsooid eT op xnoo ke satqeredui0s 
syey[Nso1 sap suuop sasodoid spoujsw ey onb JUaQUOU sMayne soy ‘UOneTNUTIs Jed opmes ouN.p udXoul 
Ny ‘plepurys sjarordsoid op yUeAIOS dS Ud JUOWIDTIOv} sonuajqo aN JUOWIATJoMjIGey yUdANed Inb jo adepuos 
op sptod sa] soisquI yUOS sayjonbsay suep sJapour uN IMs saapuoj 3S9} op sonbysye}s sap ins oindde,s 
spoyjow ey ‘oruly uoneindod oun.p om 919 e UOT]HUeYDS,] puenb s]a9powl np snuUOSUT sanowered op 
INd}99A uN.p jofns ne sasayjodAy soj JaIJU9A Inod densj}oog spoyjsur sun yuasodoid 1990g Ja JWOUNeIg 
‘onbusumu o[duoxo un,.p uaAour ne saiapuod onb Isure sameulpio sailed SaIpPUTOUL $2] Ins sayuoNy{zUT 
SOUUN sop 1OJJO,] JUONSNI]]I sy] “AQUI np guBIOTS 3sd soayeodxo sayqeueA op m9}09A UOS aonb no puRIs jso 
DIQIMWIp 93399 op dsepuos ap sprod 9] Is yueyOduN ang ynod sguUOp 9}1UN 9UN,Pp JOIAZ] ap 19]J9,] onb yuausr{Nos 
}9 IOIAZ] OP sjoyjo sap uonIsodusossp oun juasodoid sii ‘sing ‘sgigpuod saied soputour say Jed sgumso 
JUOS 9]9pOW np sonourered soy puenb (neodeyo somneul ve] op xNeUOSEIP SJUDUIZTD) SAXOUUOD IOIAI] Op saya 
sop jo nvodeyo soLeUl ek] op UoIssoidxo aun JoWUOp Jed jUsUDUIUOD ST] ‘aMeaUT] UOISsaI3a1 Jed ajgnbus,p 
SOgUUOp Sop oSATBUR | SURP SaJUONUI sau sap UOTda}9p L] op su9TqQo1d 2] JUSTPMS JURTT]e A 39 IT ‘aNsug 
‘sonbisseyo sinajeulysa say onb saovoyyo snyd 
JUDIQAR.S SINDJVWISS XNvdANOU S39 onb jUSHUOU s}eI[NSOI Sa] JOP ‘UOTe[NUMIs Jed spme oun ke jUapsoold 
}9 SOULLIRA PL] OP SIND}eWIT}Sa xNvoAnNoOU op jUasodold simayne sa] “dOULLIVA k] OP UOTeWISa,] op suIaTqQoId 
yueyodu | onb tsure ‘ase[eo op sinpgooid e] suep xnenim spiod sap x1oyo a] JWouUsIduIOD s[oNTe,] 
ap syodde sa] ‘soseyd xnop e oseuUO]]NUeYD9 | ap 33x9}U09 2] sUep adeTeO Jed UONeUMSA,| ap sUIE[qoId 
9] JUOUTeXS [epUtRS 19 ORAISY ‘ajgnbua p sanbiuysay yap oIsuINU 39 op [ole Jord a] sueq 


O1IJUINU 99 sSuUvq 
anBojeyed ne X-1.00-Z1 .N ‘epeued anbiysye}s 


Z-L dd“, .N ‘SE 10A 
6002 un! ‘ayanbua.p sanbiuyoa | 


‘y861 - 8h'6EZ ISNV ‘.S/evaeEW ‘p86 - 8y'6EZ ISNV ‘sjeva}eW 
Aseigiq pajyulid 10} sadey jo BOUaUBWdY, — ,SA8dUBIOS UOI}eW Axeigiy paul 10} 1adey Jo BOUBUBWAY — S8DUBIDS UO!}EW 
-JOJU| JO} PJEPUR}S [BEUOIVEN UBDOWY,,| ap SayewuIL SaoUaHIxe -JOJU| JO} PIBPUeYS JeUOI}ENY UBDAWY JO S}U@WAaUINbel 
xne puodas uoljeayqnd ajuasaid e| suep asin saided a4 WNWIuIW ay} sJaawW UOI}eoIqGnd siuj ul pasn seded aul 


anZoyejvo Nv X-[00-ZI oN ‘epeueD onbysnes 


Crt eer cccccccccccccccccncccccccccc cc ccccccccnccces ces enecee sees eres eeeec esse see eees er eesee eens es eesseneseressseeeeesecerccscsessssees ayens siqiimnbe aseuuo][nueyoY 
yOANeYD sumneypiny 


LOI Ane e eee e eee n erence eee e ease e ee eee essen s esses e sess sess seeseseeseeessesssesessesssssseses sajonbua xne asuoda1 eI op MAN UasoIdos PI op smMoyesoIpuy 
waysyy}eg ayer 39 Uaqqosd stuuey ‘UajnoyosS Aug 


16°" axa[dutoo aseuuo][Hueyoe ,p ueyd un mod MaMmadIAIO Ul, p dTTIQeUeA e] op UONLIOIdIOIUI | Ip 19 UOTTUTFOp eT 9q 
LIYe’] eyped 39 Jo]qeD poLysarg 


IL occ c cece ccccee cece cscee cece cece eee eter eee eseee esse ssseceeeeeseesesesceceresereeescsesesecesooes a]Jonprsal dyyeuuOsTes Ry] Inod onbiyjourered uou 1soy un, 
UPTOH 09S jo AoIAo] JoyON 


LG eae aa aaa aaa Cee soley LOUTU uoneindod op sodnoid So] SUBP SIPETeUI-sOURINSsSe sues souuosiod 
op uonodoid ey ap sanburduns jo sanbryorersty souudisaAeq soutewop s}jed inod suoneumsy 


suosleg “J ue A 39 Jjozyey UOIAT ‘THIeP BIVIQede | ‘eyUIS Iquiey ‘Wry oUTeEG “YsoyH Arye 


Iv Soe meee reece ne ene e esses ees e renee ease eee cease reese ees sees sees es esse esses sess sess ease eases sees eseeeraesssesssesssssessseses olyd, | op WIA J op xne10199]9 sasepuos 
xne uoneoydde sade yayduooul s9a.Ud sTqnop e sdUNBUTIUOD op Nea]qe} UN nod sauUdIsaheq sopomd|A 
yieg Sunsno X 39 loyD UoOM Ter ‘TOYD 3unas-og 


62" ah, ae Gk ajanbua,.p ssouuop sap sed & sasayjodAy,p $}sa} sa] tnod anbneid densyoog aspoyjsur sup) 
1990g eIyUAZ Jd yUOUNRdg sIosueIJ-UeoLr 


LI errr errr ere rere rr rrr ere rere er eee eee eee ee ee eee eee ee ee ee ee eee eee ee eee ee ee asepuos op sprod Sd] Jed solapuod IOIAD] op Se) 8) 10 neadeyo JOUR 
juelTeA preyory 3 VT nyzuers 


Saclay Sale OW sega pee aro osvjeo Jed smayewnsa daAe saseyd xnop e aseuUOT]HUeYSE | Inod a8esIA nedAnou Up 
[epules AHA-LeD 19 CBAs SY  JOWIA 


SIIINSIY SapHAV 


piercer RAAME S clmaies. inistt Sicily seine. uate A ed se, uametuaG oe lan OUT ate eee 
SITIVU SIP I[qe 
6007 unt *] orQUINU ‘CE StNTOA 


epeuey onbnysyeis Jed a9}Ipa nao su) 
dxj9nbus.p sonbruyso |, 


"BOOS UBDIEIS MMM : BPLULD NbYSNEIS Ip JOUIOJUI O}IS 9] INS so[qruodstp Juos sanbruoyse]9 

SUOISIOA Sd “99Q9NQ) Np SUdIONSHe}sS 19 SOUUDIONSYEIS Sop UONLIOOSSY | 19 PpeUuRD Np onbysyeIg 9191006 PR] ‘Yoreasay uoruIdDO 
oqng Joy uoyeloossy ueououry,| ‘sjlgnbugq,p susioysneig op s[euONeUIOJU] UOTRIOOSSY | ‘UOneIDOssy ]eoNsNRIS UPOLIOUY,| 
op SoIqUIsUT xne Yayjo jso ype xud up) ‘(Somefduioxo 7x GOI) VO $07 ‘shed sonne ‘(sonefduioxe 7x $9) WO $ZI stun 
-S}eyq : epeuRD Np Md1U9}x9,] B sIOAUD xne yUonbr{dde,s someyuourstddns uostesAl] op stey Sad “SOUUDIPeULD dJUdA op Saxe} So] sed 
snjout,u xud aq] ‘aouue sed ya ¢ g¢ ap 3S9 (anSojeieo ne GqX-100-Z1 oN) aanbua p sanbiuysay ap asgutuduit uolsi9a e] op xud a] 
yusWIIUUOGY 


‘(80°03 UBDIEIS'MMM) GOM SUIS I] INS NO sNAdI LI SuURP SagjUasaid SSATOOIIp Say JayNsuoo 

ZO][INSA eULIOJ d[ IMs suOTONAsUI so] Mog (9LO VIM ‘epeueg ‘(ouRjuO) “emeno ‘Aosuun] Wg np speuswiolg QS] “epeued 
onbysneig “esos uedje\s@)sy) ‘Joyo ua Mojoepel ne pIOAA Ud JUoWETQeIgJoId 39 onbruoHa]9 JeULIOJ Ud ste[sue ud no siesuey 
ud MmusAred ole} 9] & SOIAUT JOS s[dTUIe UN oNTeIed oMey JULMSOp sone so] ‘Ue, SIOJ xnop sat{qnd ysa ajanbua.p sanhiuysa] 
INA BI ANOd $9}x9} ap UOT}BJUISIIg 


‘epeurd onbysyeis op IU UOOepal Sp 9}TWIOD Np Sa][99 JUSUTOITeSs3999U Sed JUOS dU dNAAI BI SURP SasTtUD suOTUIO say 39 34X24 
Ind] 9p nusyuod np soqesuodsal yUSINIWop sMone so] stew ‘onbHLIO ouN kv STUMOS JUOIS SI[dIWe Sd] SNO] “saT[991 SagUUOP Sap kB 
soonbijdde no saguuop ap 93997109 eB] mod sogsI[HNn 939 JO Inb sapoyjsuT op UOVENyRAY,| & J2 UONLIOGR]I,| & 99pIODOR }S9 oJaT[NOTped 
souryodull sup) ‘sgstyeiguas soulajsks op yuowloddojaagp 9] 39 SoguUOp op asAyeue,p Jo UOTJeUIT}Sa,p sapoyjaW soy ‘sonbystye}s 
soguuOp op uoyedsaUI,| ‘sonbrydersoulsp sopms so] ‘uonesiTeuuOstesop ve] ‘sanbisojouoys souas sop asdyeue] ‘ajgnbua.p 
SOpoy}sU So] Ms syoIoyooI eI ‘sajgnbus sop uonenead,| ‘sajgnbus soy suep sMaLd so] ‘d}09][09 Op SopoyjsuI op joa soguUOp 
ap SddMOs sajUsIa]JIp op uoyestyn,| ‘onbyeid sspso,p sayureNuod ap juRTNossp UoyYdaou0d ap sowlaqod sa] ‘aydutoxs red ‘aumu09 
onbysye}s sustuesio un yuassoigjuI Inb sanbysyej}s sopoyjou sop sjoodse sISAIp So] Ins soporte sop aiqnd ajanbua p sanbiuysa] 


NOILOVGGY AG ANOLLITOd 
ppounD anbysyvig Bun x“ A\ 9 Jono} g-mqny *S Feed “Z “ezizey] ‘q IMogpoy ‘s Yoiq “ Juounvag “y-"¢ syurofpe sanayoepay 


ApISdaAluy] pADAadDOP] ‘KYSAL|SVZ “VW 

oojsajoy fo AisaaaiuQ “NM “OD 

AJISABAIUL) BJDIS DMO] “Id\OM “WY 

Dualg Ip Ipnjg 118ap BjIsdaaluy) “eULIA A “fA 
JaIpYONaN ap ayissaaluy) “TTTL “A 

oopdgqoy fo 4isaaaiuz) “uosduioy | ‘W 

AJISABAIUL) ISIPOYJaP UABYINOG “S34OIS “] 
Buosu0]oy fo Ajisdaa1uyZ) ‘1321S “Gq 

Ajisdaaluy) ajvjg o1yC ‘Kuseys ‘YJ 

uojdupyjnos fo AjissaaiuyZ) “WATS *"N OP “d 

Agjuay Yyoavasay uoiuidyE [DUCTION “UIIMNIYDS “f° 
SINSIIDIS YIJvaP] AOf AajuaD [DUOIIDNY “IaYUSYIS “N 
JDADT ajisdaaluyZ) “\SIATY *‘d-"T 

AJISAJAIUL) aYNG “IIyY “f£ 

IINJISUT JDINSIYDIS uv1puy ‘OeY “f° L 

AJISAAAIU) UOJapADD ‘ORY “WNL 


(SO0Z-SL6I) Y8UIS ‘CW Joys Ua Anaydep7J Ua}UY 


ymMoourY “YJ 
joe “H 
IRAOY ‘f 
souor “Y 
OUIqUIeD ‘ff saIquIdyAl 


pjAaqg] py fo (isaaaiuy “peselg ‘N'D'N 
AJisdaaAluy) MaAgaT] “WUCUWIIJIIg ‘q 
AJISABAIUL) 3JDJ§ OpvAOIOD ‘19WUI0SdQ ‘f 
Ajisdaaluy) Madgay] “UeUyeN “HD 

SIISYDIS AOGDT fo nvaing “I3|[A W'S 
ppouvy anbysinjg ‘a3{[ear’T] “dg 
punjdanpy fo Asaaaiuy) WSdf “WyRy “d 
INAAIS SINSIDIS JDANJJNIUBY JDUOYON WO “d 
yaavasay dood voyouayoy ‘yxAzidsey ‘qd 
Quy Jojsay ‘surypne “q 

ppouv, anbysyvjs ‘nojsolpiy “WW 
ppoun,y anbysioj¢ ‘ourquiey “¢ 

AJISAIAIUL]) AJDIS DMOT “IITIN WAN 
SIYSIDIG AOGvT fo nvaang ‘S$: ‘asuny{q “Tf 
snsua ay fo nvaang ‘SQ “Jamie” “d 

‘Ou IDISAY “MOU ‘W'L 

S21DOSSE S.1NI}IEPIY 


ppouvy anbysiynjg ‘jouer “H gnsajap Joys 
ud IndIVpsy 
ppouvny anbiysiyojs ‘Ieaoy ‘f JaYyd Ua AnajIepIy 


NOILOVaadde Ad ALINOOD 


(9861-SL61) 381d “UY 
(S00Z-9861) 2UOIsyoRIg (DH syuspIsgad sustouy 


a0h0Y 'G JUIPISI1g 
NOILOWYIG Ad ALINOOD 


‘Spoula] SANLTENY UT sJUdJUOD [eUIMOs Jd ‘SoI|sIWeIS 0} XOpU] JUOLIND suep S9dUdIIJ9I S3] 


I9ANOY ud nad UGC “AjISIOATUP SNUISeIG ‘AZO[OPoyJoP[ YOIeasay [e190 Jo osequieq WAS 39 spoeNsqy spoyjsp] pue A1ooy 
Jeonsneis ‘ueIonsneig Adams ay “(a0uaI0s Jo qaQ\) SBpayMouyY JO Gan IS] 24] Suep sgu0ped91 so ajanbua p sanbiuyoa 


epeur, onbysijej¢ ied a9}Ip9 onAoI our) 


ALANONA.d SANOINHOAL 


epeuey cs 


ePMEYO 
Sv00-v1.20 NSSI 
9[|ALISEWAS : PHOIPOHEd 
anBojeyeo ne @dX-100-Z1 oN 


600¢ uinr 


‘910 WLM Bpeuey ‘ouejUuO ‘emeO ‘epeuey snbijsije}s ‘aje}Ua!}9 Be] B SADIAIAS Sap UOISIAIG 
‘S8DUDI| OP 101}90,P SBDIAIBS XN 9}1J09 UO!SSIWJEd eB] JBDUBWAP PJOGE,P JUSAIOP SuNayesIIjN sey ‘UeWEIINY 
‘(s)aBed ja e0u919}91 Bp apolsed ‘o1QWNU Ja eWNIOA ‘anHoyeyeo ne o1BWNU ‘yinpoid Np Wou 
‘uoneolqnd ap seuue ‘epeueyd enbijsijejs : (nay e A |S ‘« ap ajdepy » no) eouNnosS : }INS BWW aE}ID a}e Op 
S@UUOP SEP 9dINOS Bj ‘Sed 9] JS9,9 IS ‘JUaLUEINAS SajelOJOWWOD UOU SUlj SEP B JIAUOS JOP UONE}USUNDOP ayeD 
‘yINpoid np s}iesxe SeireyUQWE|dWOD sjUeWeUBIesuUa! ep See}WI| SeyyUeNb ap UOHeLO e} 
syudwoo A ‘sasAjeue S890 ap SUOISN|NUOD ja SyeyjNseJ Sep UOI}EOIUNWIWOD B| SUBP je SasAjeue Sap SUep 
NUa}UOD Np uOlesI|I}N,| PUsIGWOD UOISS!LUJ3ad BaD ‘SeAl}eONps Sul Sap B No syeyUawWAaUIBANOH anbyod ap 
no ajjauUuOHesiueHO ‘ajjauUOSJad SYydJBYEJ BP Sul} Sap eB }JU@aINaS SapsONNB JUOS JINpOd 89 
Pp NUd}UOD Np uo!jesi|I}N,P SajqeuuOsies Slop Seq “89Uadl] sp iNaj}Us}ep Np swsiueHio,| ap snaiWE}xe,| 
e suolyesiuebio no sauuosied sap eB siwsuel} Noa yINposdai 3139 ynSd Bu YINpold 87 ‘SeAaSaI S}IOIP SNOL 


6002 ‘alsNpUj,| ap asIUI © 


epeuey anbijsieis ap ajqesuodsai ajjsiuiw aj Jed aasuojne uoRediqnd 


] OlgWNY e Gg ouMjoA e 600Z uInr 


epeuey onbosneis red 
II"P? 
ONADI OU 


ayonbus,p 
SonbDIUUDo | 


*« SUBIPEUBD XNe SBdIAIBS 
Sep JUYO » < « Ssnou ap sodojd y » snos eoob'ueD}e}s MMM A}IS a] INS sagiiqnd Issne jUOS BdIAJES Bp SSWJOU 
S81 “9ELL-€9Z-008-| sles Sues O4BWNU Ne epeued asnbijsije}s OaAe JaNbiunWWOD ZAaIINBA ‘adIAIe@S Bp SeWIOU 
$90 Bp 9IdOd 9UN JIUa}GO INO, ‘JUuaAJaSgO SeAC|dWA sa] anb ajajualjO Be] E BDIAIAS BP SAW JOU AP 9j}Op 1S9,S SwsiueHsO 
eujou ‘piehe j89 Y ‘sIO"NOD je sajqey ‘sepides seo|AJas Sep sjual|oO ses e JiuNoy e ebebue,s epeued anbijsiyejs 


9/9}U9I|9D Be] E BDIAIVS BP SOWION 


‘@SSOIPE S||9ANOU Be] }@ BuUBIOUe,| JIUINOJ SNOU ZAa\|INBA ‘assaupe,p JusWeHuUeUD UN ZajeUBIS SNOA anbsso7 


‘sasuojne sasiesqi| jo sjJuaHbe sap saudne suuosied uye 
910 VL (oWe}UO) EmenO 
ainjseg sAeuun| apeuswoid ‘gg, 
abeja 9 ‘s}yeOD-"H-"Y BIGNewWW| 


seoueul 
epeueyg anbijsie}S 9}SOd 
e0°06 ueoj}e}S@)s}e}sojul jalunoD - 
69Eb-L87-LL8-1 (sluf)-s}e}y Je epeued) Inaidosgje, - 
2199-297-008-1 (sluf-s}e}q Je epeued) suoydgje, - 


: sJUBAINS SUaAOW Sa] Jed sapUeWIWOD 9119 JNSd Bad! UOISJBA C7 


"SOUSA Sa] INS Sexe} Sa] Sed JUsUUaIGWOD 9U XIUd S87 


NVO$ 02 NVO$ OL shed seuny 
NVO$ ZL NVO$ 9 sluf-s}e}y 
jenuue JuUsWseUUOGY ase;dwaexy 


: Bpeuey np snaie}xa,| e sIOAUa xne juanbijdde;s s}ueAINs Saule}USWa|ddNs UOSIeJAl| Bp Sies} S97 


‘jenuue JUBWOEUUOGe UN JINOd NYO$ 8G ap je aulej|dweaxa,| 
NVO$ 0€ 9p Xd ne pyepuej}s sawidu! UOISIaA UB s/GlUOdsIp Issne jso anHojeyeo Ne X-10O-Zl U yNpoid ag 


*« SUOHEDI|GNY » anbiqnu ej JIsiOyd ap je eO"OH'ueD}e}S' MMM BC Q2/\ SYS BJJOU JO}ISIA OP INS |! 
‘asle|dwWaxe UN JIU8}GO INOg ‘ANbiuOs99/9 JEWJOJ SNOs jUBWA}IN}eI6 ajqiuodsip yse enBojeyeo ne X-1L00-ZL U yINpoJd 97 


JO9PUBWIWOD 9} NO jINpod 939 e 4apadce JUBWIWION 


2G22-S9S-008-1 JOdap ap sadiAlas Sap awuWeIHO1g 9} INOd Jnaidojgje | 
E€v6L-SE9-008-1 SJUSWSUBISSUSJ BP BDIAIBS 
jOdap ap sadiAsas Sap swWwesbOlg 


L8S0-LS6-E19-L Inaidoo9|e | 
9LL8-LS6-EL9-L SJUBWSUBI9SUSI BP BDIAIOS 
: XNEUOHEUJa}UI NO xnedo| sjaddy 


69EV-L8C-LZ8-1 Ineidoogje | 
6Z9L-€9E-008-1 sjuepuajusjew Sa| JNOd SuOHesiuNWIWODEI|9} Bp sjiaJedde,p jeuod|eU BdIAIAS 
9E1LL-€97-008-1 S]UBWAUBIaSUaI BP BDIAIES 


: (Sluf-s}ye}Q Je epeued) sles Sues SOJOWNN 
epeues anbijsijejS ap jeuoijeu 3De}UOD ap 91j}Uag 


: SJUBAINS 
soJsuinu xne IpeJpusA ne Ipun| Np OF Y QOL 18 OF YU g as}Ua SUOYd]9} Jed NO eD'oH UueD}e}S@)sjejsoju! e jewunoo 
Jed snou 998Ae Janbiunwwos jusweajebe ZeaAnod sno, ‘eo"ob'ued}e}S' MMM PB Ge/\ S}IS 94JOU JO}ISIA ‘epeueDg enbijsiyejS 
98P S8DIAJ9S SAP }@ S99UUOP SEP s|quUesuUa,| INS NO jINpOJd 8d ap jains ne s}uaWeUBlasUel ap BpUeWOEP 9}NO} INOd 


s}UsWaUBbIasuas SaijNe,p 411U9a}qGoO JUBWWOD 


epeury) Some atlas 


| : at 
"A es 
— 


ee 
| 
a 


djyoOnbDUJ,p 
SsonbIuUydo 


OR 


Survey 
Methodology 


Statistique 
Canada 


Statistics 
Canada 


ivi 


How to obtain more information 


For information about this product or the wide range of services and data available from Statistics Canada, 
visit our website at www.statcan.gc.ca, e-mail us at infostats@statcan.gc.ca, or telephone us, Monday to 
Friday from 8:30 a.m. to 4:30 p.m., at the following numbers: 


Statistics Canada’s National Contact Centre 
Toll-free telephone (Canada and United States): 


Inquiries line 1-800-263-1136 
National telecommunications device for the hearing impaired 1-800-363-7629 
Fax line 1-877-287-4369 


Local or international calls: 
Inquiries line 1-613-951-8116 
Fax line 1-613-951-0581 


Depository Services Program 
Inquiries line 1-800-635-7943 
Fax line 1-800-565-7757 


To access and order this product 


This product, Catalogue no. 12-001-X, is available free in electronic format. To obtain a single issue, visit our 
website at www.statcan.gc.ca and select “Publications.” 


This product, Catalogue no. 12-001-X, is also available as a standard printed publication at a price of 
CAN$30.00 per issue and CAN$58.00 for a one-year subscription. 


The following additional shipping charges apply for delivery outside Canada: 
Single issue Annual subscription 


United States CAN$6.00 CAN$12.00 
Other countries CAN$10.00 CAN$20.00 


All prices exclude sales taxes. 
The printed version of this publication can be ordered as follows: 


¢ Telephone (Canada and United States) 1-800-267-6677 


e Fax (Canada and United States) 1-877-287-4369 

¢ E-mail infostats @ statcan.gc.ca 

¢ Mail Statistics Canada 
Finance 


R.H. Coats Bldg., 6th Floor 
150 Tunney's Pasture Driveway 
Ottawa, Ontario K1A OT6 

¢ In person from authorized agents and bookstores. 


When notifying us of a change in your address, please provide both old and new addresses. 


Standards of service to the public 


Statistics Canada is committed to serving its clients in a prompt, reliable and courteous manner. To this end, 
Statistics Canada has developed standards of service that its employees observe. To obtain a copy of these 
service standards, please contact Statistics Canada toll-free at 1-800-263-1136. The service standards are 
also published on www.statcan.gc.ca under “About us” > “Providing services to Canadians.” 


Survey 
Methodology 


A journal 
published by 
Statistics Canada 


December 2009 e Volume 35 e Number 2 


Published by authority of the Minister responsible for Statistics Canada 
© Minister of Industry, 2009 


All rights reserved. This product cannot be reproduced and/or transmitted to any person or 
organization outside of the licensee’s organization. Reasonable rights of use of the content of this product are 
granted solely for personal, corporate or public policy research, or for educational purposes. 

This permission includes the use of the content in analyses and the reporting of results and conclusions, 
including the citation of limited amounts of supporting data extracted from this product. These materials are solely 
for non-commercial purposes. In such cases, the source of the data must be acknowledged as follows: 
Source (or “Adapted from’, if appropriate): Statistics Canada, year of publication, name of product, catalogue number, 
volume and issue numbers, reference period and page(s). Otherwise, users shall seek prior written permission 
of Licensing Services, Client Services Division, Statistics Canada, Ottawa, Ontario, Canada K1A OT6. 


December 2009 
Catalogue no. 12-001-XPB 
Frequency: Semi-annual 
ISSN 0714-0045 


Ottawa 


Be S2tstcs Statistique Canada 


SURVEY METHODOLOGY 
A Journal Published by Statistics Canada 


Survey Methodology is indexed in The ISI Web of knowledge (Web of science), The Survey Statistician, Statistical Theory 
and Methods Abstracts and SRM Database of Social Research Methodology, Erasmus University and is referenced in the 


Current Index to Statistics, and Journal Contents in Qualitative Methods. 


MANAGEMENT BOARD 
Chairman J. Kovar 


Past Chairmen D. Royce (2006-2009) 
G.J. Brackstone (1986-2005) 
R. Platek (1975-1986) 


EDITORIAL BOARD 


Editor J. Kovar, Statistics Canada 
Deputy Editor H. Mantel, Statistics Canada 


Associate Editors 


J.M. Brick, Westat Inc. 

P. Cantwell, U.S. Bureau of the Census 

J.L. Eltinge, U.S. Bureau of Labor Statistics 
W.A. Fuller, Jowa State University 

J. Gambino, Statistics Canada 

M.A. Hidiroglou, Statistics Canada 

D. Judkins, Westat Inc 

D. Kasprzyk, Mathematica Policy Research 

P. Kott, National Agricultural Statistics Service 


Members J. Gambino 


J. Kovar 

J. Latimer 

H. Mantel 

S. Fortier (Production Manager) 


Past Editor M.P. Singh (1975-2005) 


T.J. Rao, Indian Statistical Institute 

J. Reiter, Duke University 

L.-P. Rivest, Université Laval 

N. Schenker, National Center for Health Statistics 
F.J. Scheuren, National Opinion Research Center 
P. doN. Silva, University of Southampton 

E. Stasny, Ohio State University 

D. Steel, University of Wollongong 

L. Stokes, Southern Methodist University 


P. Lahin, JPSM, University of Maryland 
P. Lavallée, Statistics Canada 

G. Nathan, Hebrew University 

J. Opsomer, Colorado State University 
D. Pfeffermann, Hebrew University 
N.G.N. Prasad, University of Alberta 
J.N.K. Rao, Carleton University 


Assistant Editors J.-F. Beaumont, P. Dick, S. Godbout, D. Haziza, Z. Patak, S. Rubin-Bleuer and W. Yung, Statistics Canada 


M. Thompson, University of Waterloo 

Y. Tillé, Université de Neuchatel 

V.J. Verma, Universita degli Studi di Siena 
K.M. Wolter, Jowa State University 

C. Wu, University of Waterloo 

A. Zaslavsky, Harvard University 


EDITORIAL POLICY 


Survey Methodology publishes articles dealing with various aspects of statistical development relevant to a statistical agency, 
such as design issues in the context of practical constraints, use of different data sources and collection techniques, total 
survey error, survey evaluation, research in survey methodology, time series analysis, seasonal adjustment, demographic 
studies, data integration, estimation and data analysis methods, and general survey systems development. The emphasis is 
placed on the development and evaluation of specific methodologies as applied to data collection or the data themselves. All 
papers will be refereed. However, the authors retain full responsibility for the contents of their papers and opinions expressed 
are not necessarily those of the Editorial Board or of Statistics Canada. 


Submission of Manuscripts 


Survey Methodology is published twice a year. Authors are invited to submit their articles in English or French in electronic 
form, preferably in Word to the Editor, (smj@statcan.gc.ca, Statistics Canada, 150 Tunney’s Pasture Driveway, Ottawa, 
Ontario, Canada, K1A 0T6). For formatting instructions, please see the guidelines provided in the journal and on the web site 
(www.statcan.gc.ca). 


Subscription Rates 


The price of printed versions of Survey Methodology (Catalogue No. 12-001-XPB) is CDN $58 per year. The price excludes 
Canadian sales taxes. Additional shipping charges apply for delivery outside Canada: United States, CDN $12 ($6 x 2 
issues); Other Countries, CDN $20 ($10 x 2 issues). A reduced price is available to members of the American Statistical 
Association, the International Association of Survey Statisticians, the American Association for Public Opinion Research, the 
Statistical Society of Canada and |’ Association des statisticiennes et statisticiens du Québec. Electronic versions are available 
on Statistics Canada’s web site: www.statcan.gc.ca. 


Survey Methodology 
A Journal Published by Statistics Canada 
Volume 35, Number 2, December 2009 


Contents 

Ents USSG MAY, OR FI CT a NSPS AR SN re ee Oe Re ae as Al 
Waksberg Invited Paper Series 
Graham Kalton 

Methods foroversampling rare subpopulationsumsocial/surveysic ...c).N aie, Sys A Ices 125 
Regular Papers 
Andreas Quatember 

A standardization, ofrandomized response strates tes «47a, 8 kes os seme: Stee Oe oe. PRUE. laws ensues 143 
Xiaojian Xu and Pierre Lavallée 

Treatinents torlink nonresponse, inandirect Sampling 87a goes autem eg mun anieem areas ee ea cn 153 
Damiao N. da Silva and Jean D. Opsomer 

Nonparametric propensity weighting for survey nonresponse through local polynomial regression ..............:0000 165 


Jan van den Brakel and Sabine Krieg 
Estimation of the monthly unemployment rate through structural time series modelling in a rotating 


DANGIGCOSI Steer. someone ARNON, SMR Mee tee Te sR UNM mM te) Oe eNO ee fen cee Seen een teeter eat ante Mades ones Reeenttoneector dascenee> 19a, 
Li-Chun Zhang 
Estimates for small area compositions subjected to informative missing data ...........:.ceeseeeseseseeeeceeeseseeeeeeeeseeeeeeaeees 19] 


Debora F. Souza, Fernando A.S. Moura and Helio S. Migon 
Smiall-areapopnlatiompredictioniviatierarchicalimodelst e000. Lie. ME Se An aes. 203 


Jun Shao and Katherine J. Thompson 


Variance estimation in the presence of nonrespondents and Certainty Strata ..0........eceeeeeeeeseeeeeeeeeeeeeeeseeeseeeeetseseeeeeeees 215 
John Preston 
Rescealed. bootstrap tos stratified multistage sampling tccen see fret. axe tave sernetaaen seer sae Sis Soe tee ti2.... wey 


Donsig Jang and John L. Eltinge 
Use of within-primary-sample-unit variances to assess the stability of a standard 
Gesion-based vanancevestiMatOn sss we cee ent aa ced naan see Ps eee as Aran Bann eR ties ss: 239 


Zilin Wang and David R. Bellhouse 
Semipatamethic reeression:model:for complex isunve yoda tase geen. RA ....3. al. Ree aN en Rn Serer ccsa nant 247 


ACKNOWICCZCMEN ES Hi ata deere irri ate ee tne a See anh sata ov ed he meee hie erent... 261 


Statistics Canada, Catalogue No. 12-001-X 


The paper used in this publication meets the minimum Le papier utilisé dans la présente publication répond aux 
requirements of American National Standard for Infor- exigences minimales de I’“American National Standard for Infor- 
mation Sciences — Permanence of Paper for Printed Library mation Sciences” — “Permanence of Paper for Printed Library 
Materials, ANSI Z39.48 - 1984. Materials”, ANSI Z39.48 - 1984. 


Survey Methodology, December 2009 121 
Vol. 35, No. 2, pp. 121-122 
Statistics Canada, Catalogue No. 12-001-X 


In this issue 


This issue of Survey Methodology opens with the ninth paper in the annual Waksberg Award 
invited paper series in honour of Joseph Waksberg’s contributions to the theory and practice of 
survey methodology. The editorial board would like to thank the members of the selection committee 
— Bob Groves, chair, Leyla Mohadjer, Daniel Kasprzyk and Wayne Fuller — for having selected 
Graham Kalton as the author of this year’s Waksberg Award paper. 

In his paper entitled “Methods for oversampling rare subpopulations in social surveys” Kalton 
gives an overview of methods for sampling rare populations, what Kish called minor domains. After 
discussing general issues he describes several different methods including screening, stratification, 
two-phase sampling, multiple frames, multiplicity sampling, location sampling, and accumulating 
samples over time. He discusses the advantages and disadvantages of each method, and gives many 
examples of their use in surveys. In practice a combination of approaches is often used. 

Randomized response strategies are often used in order to reduce nonsampling errors such as 
nonresponse and measurement errors. They can also be used in the context of statistical disclosure 
control for public use microdata files. In his paper, Quatember proposes a standardization of 
randomized response techniques. The statistical properties of the standardized estimator are derived. 
He applies the proposed method to a survey on academic cheating behaviour. 

Xu and Lavallée consider the problem caused by link nonresponse when using the generalized 
weight share method in indirect sampling. Indirect sampling is used when selecting samples from a 
population that is not the target population of interest but is related to it. Biased estimates may occur 
when it is not known that a unit in the sampling population is related to a unit in the target population. 
The authors propose several weight adjustments to overcome the issue of link nonresponse. 

In the context of unit nonresponse, the weights of the respondents are often adjusted by the inverse 
of the estimated response probability. Da Silva and Opsomer propose to estimate the response 
probabilities using local polynomial regression. Results of a simulation study are presented 
confirming the good performance of the proposed method. 

In their paper, Van den Brakel and Krieg consider a multivariate structural time series model that 
accounts for the design of the Dutch Labour Force Survey. The model is used to estimate the 
unemployment rates. An empirical investigation demonstrates that the proposed model results in a 
significant increase in accuracy. 

Zhang considers estimation of cross-classifications where one margin of the cross-classification 
corresponds to small areas and where non-response varies from area to area. He develops a double 
mixed model approach that combines the fixed effects and random area effects of the small area 
model with the random effects from the missing data mechanism. The associated conditional mean 
squared error of prediction is approximated in terms of a three-part decomposition, corresponding to 
a naive prediction variance, a positive correction that accounts for the hypothetical parameter 
estimation uncertainty based on the latent complete data, and another positive correction for the extra 
variation due to the missing data. 

Souza, Moura and Migon propose a Bayesian small area estimation application using growth 
models that account for hierarchical and spatial relationships. They use this approach to obtain 
population predictions for the municipalities not sampled in the Brazilian Annual Household Survey 
and to increase the precision of the design-based estimates obtained for the sampled municipalities. 

Shao and Thompson investigate the problem of variance estimation when a weight adjustment is 
applied to deal with nonresponse in stratified business surveys. They derive two consistent 
linearization variance estimators under weak assumptions. Naive jackknife variance estimators do not 
work well unless the sampling fraction is negligible, which is not the case when there are certainty 
strata. They propose a modified jackknife variance estimator that is consistent even when there are 
certainty strata but the non-certainty strata must not have a large sampling fraction. They evaluate 
their variance estimators empirically using real data and a simulation study. 


ZZ . In this issue 


In his paper, Preston investigates the bootstrap variance estimation for multistage designs when 
units are selected using simple random sampling without replacement at each stage. He proposes an 
extension to the commonly used rescaled bootstrap estimator that assumes with replacement 
sampling or negligible sampling fractions at the first stage. The proposed estimator is compared with 
the rescaled and Bernoulli bootstrap estimators. 

Jang and Eltinge address the problem of estimating degrees of freedom values from stratified 
multistage designs when a small number of primary sampling units (PSUs) are selected per stratum. 
Due to the small number of PSUs selected, the traditional Satterthwaite-based degrees of freedom can 
be a severe underestimate. In their paper, they propose an alternative estimator of the degrees of 
freedom that uses the within PSU variances to provide auxiliary information on the relative 
magnitudes of the overall stratum-level variances. The proposed method is illustrated using data from 
the National Health and Nutrition Examination Survey (NHANES). 

The article by Wang and Bellhouse explores an application of nonparametric regression 
techniques to study the relationship between the response variable and covariates, as well as 
prediction using auxiliary information in the context of complex surveys. The work is an extension of 
Bellhouse and Stafford (2001) that used a simple nonparametric regression function to the case of 
several independent variables, including indicator variables that often appear in regression analysis 
using survey data. 

And finally, we are pleased to inform readers and authors that Survey Methodology will 
shortly be covered by SCOPUS in the Elsevier Bibliographic Databases starting with the June 
2008 issue. 


Harold Mantel, Deputy Editor 
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Waksberg Invited Paper Series 


The journal Survey Methodology has established an annual invited paper series in honour of Joseph 
Waksberg, who has made many important contributions to survey methodology. Each year a prominent 
survey researcher is chosen to author an article as part of the Waksberg Invited Paper Series. The paper 
reviews the development and current state of a significant topic within the field of survey methodology, and 
reflects the mixture of theory and practice that characterized Waksberg’s work. The author receives a cash 
award made possible by a grant from Westat, in recognition of Joe Waksberg’s contributions during his 
many years of association with Westat. The grant is administered financially by the American Statistical 
Association. 


Waksberg Award Winners: 


Gad Nathan (2001) 
Wayne A. Fuller (2002) 
Tim Holt (2003) 

Norman Bradburn (2004) 
J.N.K. Rao (2005) 
Alastair Scott (2006) 
Carl-Enk Sarndal (2007) 
Mary Thompson (2008) 
Graham Kalton (2009) 
Ivan Fellegi (2010) 


Nominations: 


The author of the 2011 Waksberg paper will be selected by a four-person committee appointed by Survey 
Methodology and the American Statistical Association. Nominations of individuals to be considered as 
authors or suggestions for topics should be sent to the chair of the committee, Daniel Kasprzyk, by email to 
DKasprzyk@Mathematica-Mpr.com. Nominations and suggestions for topics must be received by 
February 28, 2010. 


2009 Waksberg Invited Paper 
Author: Graham Kalton 


Graham Kalton is Chairman of the Board of Directors and a Senior Vice President at Westat. He has a 
title of Research Professor in the Joint Program in Survey Methodology at the University of Maryland. 

Dr. Kalton has wide-ranging interests in survey methodology, and has published on several aspects of 
the subject, including sample design, nonresponse and imputation, panel surveys, question wording, 
and coding.He is a Fellow of the American Association for the Advancement of Science, a Fellow of 
the American Statistical Association, a National Associate of the National Academies, and an elected 
member of the International Statistical Institute. He delivered the annual Morris Hansen lecture in 
2000. 
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Members of the Waskberg Paper Selection Committee (2009-2010) 


Daniel Kasprzyk (Chair), Mathematica Policy Research 
Wayne A. Fuller, Jowa State University 

Elizabeth A. Martin 

Mary Thompson, University of Waterloo 
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Methods for oversampling rare subpopulations in social surveys 


Graham Kalton ! 


Abstract 


Surveys are frequently required to produce estimates for subpopulations, sometimes for a single subpopulation and 
sometimes for several subpopulations in addition to the total population. When membership of a rare subpopulation (or 
domain) can be determined from the sampling frame, selecting the required domain sample size is relatively straightforward. 
In this case the main issue is the extent of oversampling to employ when survey estimates are required for several domains 
and for the total population. Sampling and oversampling rare domains whose members cannot be identified in advance 
present a major challenge. A variety of methods has been used in this situation. In addition to large-scale screening, these 
methods include disproportionate stratified sampling, two-phase sampling, the use of multiple frames, multiplicity sampling, 
location sampling, panel surveys, and the use of multi-purpose surveys. This paper illustrates the application of these 


methods in a range of social surveys. 


Key Words: Sample allocation; Screening; Disproportionate stratified sampling; Two-phase sampling; Multiple 
frames; Location sampling; Panel surveys; Multi-purpose surveys. 


1. Introduction 


I feel very privileged to have been invited to present this 
year’s paper in the Waksberg Invited Paper Series, a series 
that honors Joe Waksberg for his numerous contributions to 
survey methodology. I was extremely fortunate to have had 
the opportunity to work with Joe at Westat for many years 
and, as did many others, I benefited greatly from that expe- 
rience. When faced with an intractable sampling problem, 
Joe had a flair for turning the problem on its end and 
producing a workable solution. Since the problem often 
concemed the sampling of rare populations, I have chosen 
to review methods for sampling rare populations for this 
paper. 

One of the major developments in survey research over 
the past several decades has been the continuously esca- 
lating demand for estimates for smaller and smaller sub- 
classes (subpopulations) of the general population. This 
paper focuses on those subclasses — termed domains — that 
are planned for separate analysis at the sample design stage. 
Some examples of domains that have been taken into 
account in the sample designs of various surveys include a 
country’s states or provinces, counties or districts; racial/ 
ethnic minorities; households living in poverty; recent 
births; persons over 80 years of age; recent immigrants; gay 
men; drug users; and disabled persons. When the domains 
are small (also known as rare populations), the need to 
provide adequate sample sizes for domain analysis can 
create major challenges in sample design. This paper 
reviews the different probability sampling methods that are 
used to generate samples for estimating the characteristics 
of rare populations with required levels of precision. 
Sampling methods for estimating the size of a rare 


population are not explicitly addressed, although similar 
methods are often applicable. However, capture-recapture 
and related methods are not addressed in this paper. 

An important issue for sample design is whether the aim 
of a survey is to produce estimates for a single domain or 
many domains. Although much of the literature on the 
sampling of rare populations discusses sample designs for a 
single rare domain (e.g., drug users), in practice surveys are 
often designed to produce estimates for many domains (e.g., 
each of the provinces in a country or several racial/ethnic 
groups). The U.S. National Health and Nutrition Exam- 
ination Survey (NHANES) is an example of a survey 
designed to produce estimates for many domains, in this 
case defined by age, sex, race/ethnicity and low-income 
status (Mohadjer and Curtin 2008). In sample designs that 
include many domains, the domains may be mutually 
exclusive (e.g., provinces or the cells of the cross- 
classification of age group and race/ethnicity) or they may 
be intersecting (e.g., domains defined separately by age 
group and by race/ethnicity). 

The size of a domain is a key consideration. Kish (1987) 
proposed a classification of major domains of perhaps 10 
percent or more of the total population, for which a general 
sample will usually produce reliable estimates; minor 
domains of | to 10 percent, for which the sampling methods 
in this paper are needed; mini-domains of 0.1 to | percent, 
estimates for which mostly require the use of statistical 
models; and rare types comprising less than 0.01 percent of 
the population, which generally cannot be handled by 
survey sampling methods. Many surveys aim to produce 
estimates for some major domains, some minor domains 
and occasionally even some mini-domains. 
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Since the sample sizes for most surveys are sufficient to 
produce estimates of reasonable precision for major 
domains, there is generally no need to adopt the kinds of 
oversampling procedures reviewed in this paper. However, 
there are some important design features that should be 
considered. It is, for example, valuable to take major 
domains into account in creating the strata for the survey. 
This consideration is of particular importance with 
geographically defined domains and multistage sampling. If 
a geographic domain is not made into a design stratum, the 
number of primary sampling units (PSUs) selected in that 
domain is a random variable; the sampled PSUs in strata 
that cut across the domain boundaries may or may not be in 
the domain, creating problems for domain estimation. It is 
also valuable to have a sizable number of sampled PSUs in 
each geographical domain in order to be able to compute 
direct variance estimates of reasonable precision, implying 
the need to spread the sample across a large number of 
PSUs. At the estimation stage, it is preferable, where 
possible, to apply nonresponse and noncoverage post- 
stratification-type adjustments at the domain rather than the 
national level. Singh, Gambino and Mantel (1994) and 
Marker (2001) discuss design issues and Rao (2003, pages 
9-25) discusses estimation issues for major domains. Major 
domains will receive little attention in this paper. 

At the other end of the size continuum, even with the use 
of special probability sampling methods, the sample sizes 
possible for most surveys are not large enough to produce 
standard design-based, or direct, estimates of characteristics 
for multiple domains when many of the domains are mini- 
domains or rare types. An obvious exception is a national 
population census, but censuses too have their limitations. 
Since they are conducted infrequently (in many countries 
only once a decade), their estimates are dated — a particular 
concern for mini-domains, which can experience rapid 
changes. Also, the content of a census must be severely 
limited in terms of the range of topics and depth of detail. 
Very large continuous surveys such as the American 
Community Survey (U.S. Census Bureau 2009a; Citro and 
Kalton 2007), the French rolling census (Durr 2005) and the 
German Microcensus (German Federal Statistical Office 
2009) have been developed to address the need for more up- 
to-date data for small domains, but a restriction on content 
remains (although the content of the German Microcensus 
does vary over time). Other exceptions occur at the border 
between mini-domains and minor domains. For example, 
since 2007 the Canadian Community Health Survey has 
provided estimates on the health status of the populations of 
each of Canada’s 121 health regions based on an annual 
survey of around 65,000 persons aged 12 and over, with the 
production of annual and biennial data files (Statistics 
Canada 2008). By combining the samples across multiple 
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years, researchers are able to produce estimates for rare 
populations of various types. 

In general, however, the maximum sample size possible 
for a survey on a specific topic is not adequate to yield a 
large set of mini-domain estimates of acceptable precision. 
Yet policy makers are making increasing demands for local 
area data at the mini-domain level. This demand for esti- 
mates for mini-domains, mainly domains defined at least in 
part by geographical administrative units, is being addressed 
by the use of statistical modeling techniques, leading to 
model-dependent, indirect, small area estimates. Thus, for 
example, the U.S. Census Bureau’s Small Area Income and 
Poverty Estimates program produces indirect estimates of 
income and poverty statistics for 3,141 counties and 
estimates of poor school-age children for around 15,000 
school districts every year, based on data now collected in 
the American Community Survey and predictor variables 
obtained from other sources available at the local area level, 
such as tax data (U.S. Census Bureau 2009b). A compre- 
hensive treatment of indirect estimation using small area 
estimation techniques, a methodology that falls outside the 
scope of this paper, can be found in Rao (2003). 

Apart from location sampling, discussed in Section 3.6, 
this paper also does not address the various methods that 
have been developed for sampling other types of muini- 
domains of much interest to social researchers and epi- 
demiologists, domains that are often “hidden populations” 
in that the activities defining them are clandestine, such as 
intravenous drug use (Watters and Biernacki 1989). A range 
of methods has been developed under the assumption that 
the members of the mini-domains know each other. The 
broad class of such designs is termed link-tracing designs 
(see the review by Thompson and Frank 2000). They are 
adaptive designs in that the units are selected sequentially, 
with those selected at later stages dependent on those 
selected earlier (Thompson and Seber 1996; Thompson 
2002). 

Snowball sampling was one of the early methods of an 
adaptive, chain-referral sample design. It starts with some 
initial sample of rare domain members (the seeds), and they 
in turn identify other members of the domain. While it bears 
a resemblance to network (multiplicity) sampling (described 
in Section 3.5), snowball sampling lacks the probability 
basis of the latter technique, i.e., known, non-zero, selection 
probabilities for all members of the domain. A version of 
snowball sampling has been termed respondent-driven 
sampling (RDS) (Heckathom 1997, 2007). Volz and 
Heckathorn (2008) develop a theory for RDS that is based 
on four assumptions: (1) that respondents know how many 
members of the network are linked to them (the degree); (2) 
that respondents recruit others from their personal network 
at random; (3) that network connections are reciprocal; and 
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(4) that recruitment follows a Markov process. The need for 
these modeling assumptions for statistical inference is the 
difference between chain-referral sample designs and the 
conventional probability sample designs used in surveys 
which do not need to invoke such assumptions. It is 
apparent that RDS is appropriate only for mini-domains for 
which clear networks exist. The method is used mainly in 
local area settings, but Katzoff, Sirken and Thompson 
(2002) and Katzoff (2004) have suggested that the seeds 
could come from a large-scale survey, such as the U.S. 
National Health Interview Survey. 

This paper focuses on the use of probability sampling 
methods to produce standard design-based, or direct, 
estimates for characteristics of rare populations, building on 
previous reviews (e.g., Kish 1965a; Kalton and Anderson 
1986; Kalton 1993a, 2003; Sudman and Kalton 1986; 
Sudman, Sirken and Cowan 1988; and Flores Cervantes and 
Kalton 2008). Much of the literature deals with the sampling 
issues that arise when the rare population is the sole subject 
of study. However, as noted above, surveys are often 
required to produce estimates for many different domains as 
well as for the total population. Section 2 reviews the design 
issues involved when the survey has design objectives for 
multiple domains whose members can be identified from 
the sampling frame. The main part of the paper, Section 3, 
provides a review of a range of methods that have been used 
to sample rare populations whose members cannot be 
identified in advance. The paper ends with some concluding 
remarks in Section 4. 


2. Miulti-domain allocations 


The issue of sample allocation arises when a survey is 
being designed to produce estimates for a number of 
different domains, for subclasses that cut across the 
domains, as well as for the total population. In most 
applications, domains vary considerably in size with at least 
some of them being rare domains. 

Assume that there are H mutually exclusive and 
exhaustive domains that are identified on the sampling 
frame. Under the commonly made assumptions that the 
variance of an estimate for domain / can be expressed as 
V/n, and that survey costs are the same across domains, 
the optimum allocation for estimating the overall population 
mean is n, « W,, where W, is the proportion of the 
population in domain hf. Assuming that the domain 
estimates are all to have the same precision, the optimum 
allocation is n, =n/H_ for all domains. These two 
allocations are in conflict when the W, vary greatly, as 
often occurs when the domains are administrative areas of 
the country, such as states, provinces, counties or districts. 
In such cases, adopting the optimum allocation for one 


T2i, 


objective leads to a serious loss of precision for the other. 
However, a compromise allocation that falls between the 
two optimum allocations often works well for both 
objectives. 

Several compromise solutions exist. One, proposed by 
Kish (1976, 1988), is to determine the domain sample sizes 
by the following formula: 


n, « JIW2 +(1-I)H”, 


where J and (1 — /) represent the relative importance of the 
national estimate and the domain (e.g., administrative 
district) estimates, respectively. If J = 1, the allocation is a 
proportionate allocation, as optimum for the national 
estimate, whereas if J = 0, the allocation is an equal 
allocation, as optimum for the domain estimates. The choice 
of J is highly subjective, but I have found that J = 0.5 is 
often a good starting point, after which a careful review of 
the allocation can lead to modifications. Bankier (1988) has 
proposed a similar compromise solution, termed a power 
allocation. Applied to the current example, the domain 
sample sizes are determined from n, « W,’, where q isa 
power between 0 (equal allocation) and 1 (proportionate 
allocation). As an example, the 2007 Canadian Community 
Health Survey was designed to attach about equal impor- 
tance to the estimates for provinces and health regions. The 
sample allocation to a province was based on its population 
size and its number of health regions. Within a province, the 
sample was allocated between health regions using the 
Bankier allocation with g =0.5 (Statistics Canada 2008). 

A limitation to the Kish and Bankier procedures is that 
they may not allocate sufficient sample to small domains to 
produce estimates at the required level of precision. This 
limitation can be addressed by revising the initial allocations 
to satisfy precision requirements. An alternative approach 
addresses this limitation directly: the allocation is deter- 
mined by fixing a core sample that will satisfy one of the 
objectives and then supplementing that sample as needed to 
satisfy the other objective. Singh, Gambino and Mantel 
(1994) describe such a design for the Canadian Labour 
Force Survey, with a core sample to provide national and 
provincial estimates and, where needed, supplemental 
samples to provide subprovincial estimates of acceptable 
precision. 

The Kish and Bankier schemes assume that the same 
precision level is required for all small domains. Longford 
(2006) describes a more general approach in which 
‘inferential priorities’ P, are assigned to each domain d. As 
an example, he proposes setting the priorities as P, = N°, 
where N,, is the population size of domain d and a is a 
value chosen between 0 and 2. The value a =0 corresponds 
to the Kish and Bankier equal domain sample size assump- 
tion and a =2 corresponds to an overall proportionate 
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allocation. An intermediate value of a attaches greater priority 
to larger domains. Longford also extends the approach to 
incorporate an inferential priority for the overall estimate. 

A more general approach to sample allocation is via 
mathematical programming, as has been proposed by a 
number of researchers (see, for example, Rodriguez Vera 
1982). This approach can accommodate unequal variances 
across domains, intersecting domains, and multiple 
estimates for each domain. The U.S. Early Childhood 
Longitudinal Study — Birth Cohort (ECLS-B) provides an 
example with intersecting domains, with the sample 
selected from birth certificate records that contained the 
requisite domain information. There were 10 domains of 
interest for the ECLS-B: births classified by race (5 
domains), birth weight (3 domains) and twins or non-twins 
(2 domains). The approach adopted first determined a 
minimum effective sample size (i.e., the actual sample size 
divided by the design effect) for each domain. With the 30 
cells of the cross-classification of birth weight, race/ 
ethnicity and twin/non-twin treated as strata, an allocation of 
the sample across the strata was then determined to 
minimize the overall sample size while satisfying the 
effective sample size requirements for all the domains 
(Green 2000). 

When there are multiple domains of interest and multi- 
stage sampling is to be used, a variant of the usual measure 
of size for probability proportional to size (PPS) sampling 
can be useful for controlling the sample sizes in the sampled 
clusters (PSUs, second-stage units, efc.), provided that 
reasonable estimates of the domain population sizes are 
available by cluster. The requirements that all sampled 
clusters have approximately the same overall subsample 
size and that sampled units in each domain have equal 
probabilities of selection can both be met by sampling the 
clusters with standard PPS methods, but with a composite 
measure of size that takes account of the differing sampling 
rates for different domains (Folsom, Potter and Williams 
1987). As an example, in a survey of men in English 
prisons, the desired sampling fractions were | in 2 for civil 
prisoners (C), 1 in 21 for “star” prisoners who are normally 
serving their first term of imprisonment (S) and | in 45 for 
recidivists (R). Prisons were selected at the first stage of 
sampling, with prison i being selected with probability 
proportional to its composite measure of size R, + 
2.2S, + 20.3C,, where the multipliers are the sampling 
rates relative to the rate for recidivists (Mormis 1965, pages 
303-306). 


3. Methods for oversampling rare domains 


The main focus of this paper is on the use of probability 
sampling methods to produce standard design-based, or 
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direct, estimates for characteristics of rare populations, often 
minor domains in Kish’s terminology. As preparation for 
the subsequent discussion, it will be useful to note some 
features of different types of rare populations that, together 
with the survey’s mode of data collection, are influential in 
the choice of sampling methods that can be applied to 
generate required sample sizes for all domains. Some 
important features for consideration are summarized below: 


— Isa separate frame(s) available for sampling a rare 
population? Can those sampled be located for data 
collection? How up-to-date and complete is the 
frame? If an existing up-to-date frame contains 
only the rare population (with possibly a few other 
listings) and provides almost complete coverage, 
then sampling can follow standard methods. If no 
single frame gives adequate coverage but there are 
multiple frames that between them give good 
coverage, issues of multiple routes of selection 
arise (Section 3.4). 

— Is the rare population concentrated in certain, 
identifiable parts of the sampling frame, or is it 
fairly evenly spread throughout the frame? If it is 
concentrated, disproportionate stratification can be 
effective (Section 3.2). 

— If a sample is selected from a more general 
population, can a sampled person’s membership in 
the rare population be determined inexpensively, 
such as from responses to a few simple questions? 
If so, standard screening methods may be used 
(Section 3.1). If accurate determination requires 
expensive procedures, such as medical examina- 
tions, a two-phase design may be useful (Section 
3.3). A related issue is whether some members of a 
rare population consider their membership to be 
sensitive; the likelihood that members may be 
tempted to deny their membership may influence 
the choice of survey administration mode and 
other aspects of screening. 

— Are members of the rare population readily 
identified by others? If so, some form of network, 
or multiplicity, sampling may be useful (Section 
Se he 

— Are members of the rare population to be found at 
specific locations or events? If so, location 
sampling may be useful (Section 3.6). 

— Is the rare population defined by a constant 
characteristic (e.g., race/ethnicity) or by a recent 
event (e.g., a hospital stay)? The distinction 
between these two types of characteristics is 
important in considering the utility of panel 
surveys for sampling rare populations (Section 
yp 
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The following sections review a range of methods for 
sampling rare populations. Although the methods are 
discussed individually, some are interrelated and, in 
practice, a combination of methods is often used. 


3.1 Screening 


Some form of screening is generally needed when the 
sampling frame does not contain domain identifiers. This 
section considers a straightforward application of a screening 
design in which a large first-phase sample is selected to 
identify samples of the members of the domains of interest, 
without recourse to the techniques described in later 
sections. The first-phase sample size is the minimum sample 
size that will produce the required (or larger) sample sizes 
for all of the domains. The minimum first-phase sample size 
is determined by identifying the required sample size for 
one of the domains, with all of the sample members of that 
domain then being included in the second-phase sample. 
Subsamples of other domains are selected for the second- 
phase sample at rates that generate the required domain 
sample sizes. If the survey is designed to collect data for 
only a subset of the domains (often only one domain), then 
none of the members of the other domains is selected for the 
second-phase sample. 

Since a very large screening sample size is needed to 
generate an adequate domain sample size when one (or 
more) of the domains of interest is a rare population, the 
cost of screening becomes a major concern. In addition to 
the sampling methods discussed in later sections, there are 
several strategies that can be employed to keep costs low: 


— Use an inexpensive mode of data collection, such 
as telephone interviewing or a mail questionnaire, 
for the screening. The second-phase data collection 
may be by the same mode or a different mode. 

— When possible and useful, permit the collection of 
screening data from persons other than those 
sampled. For example, other household members 
may be able to accurately report the rare popula- 
tion status of the sampled member. See the 
discussion below and also Section 3.5 on multi- 
plicity sampling. 

— When screening is carried out by face-to-face 
interviewing in a multistage design, it is efficient 
to select a large sample size in each cluster. 
Compact clusters can also be used. Costs are 
reduced, and the precision of domain estimates is 
not seriously harmed because the average domain 
sample sizes in the clusters will be relatively small. 


One possible means of reducing screening costs is to 
share the costs across more than one survey. For instance, 
the child component of the ongoing U.S. National 
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Immunization Survey (NIS) is a quarterly telephone survey 
that screens households with landline telephone numbers to 
locate children aged 19 to 35 months, in order to ascertain 
vaccination coverage levels (Smith, Battaglia, Huggins, 
Hoaglin, Roden, Khare, Ezzati-Rice and Wright 2001; U.S. 
National Center for Health Statistics 2009b). The NIS large- 
scale screening is also used to identify members of domains 
of interest for the State and Local Area Integrated 
Telephone Survey (SLAITS) program, which addresses a 
variety of other topics over time (U.S. National Center for 
Health Statistics 2009c). When sharing screening costs 
across a number of surveys, it is advantageous if the 
domains for the surveys are fairly disjoint sets in order to 
minimize the problems associated with screening some 
respondents into more than one survey. 

When no one is at home to complete a face-to-face 
screening for a household, it may be possible to obtain 
information from knowledgeable neighbors as to whether 
the household contains a member of the rare population 
(e.g., a child under 3 years of age). This approach (which is 
used in NHANES) can appreciably reduce data collection 
costs when a large proportion of the households do not 
contain members of the rare population. However, there is a 
danger that the approach may result in undercoverage; some 
protection is provided by requiring that, if the first neighbor 
interviewed indicates that the household does not include a 
member of the rare population(s), the other neighbor is also 
interviewed. Ethical issues also must be considered, 
particularly for the identification of rare populations that are 
sensitive in nature. 

An extension of the approach of collecting screening 
information from neighbors is known as_ focused 
enumeration. This technique, which is a form of multiplicity 
sampling (see Section 3.5), involves asking the respondent 
at each sampled, or “core”, address about the presence of 
members of the rare population in the n neighboring 
addresses on either side. In essence, the sample consists of 
2n +1 addresses for each core address. If the respondent is 
unable to provide the screening information for one or more 
of the linked addresses, then the interviewer must make 
contact at another address. Focused enumeration has been 
used with n =2 in the British Crime Survey (Bolling, Grant 
and Sinclair 2008) and the Health Survey of England (Erens, 
Prior, Korovessis, Calderwood, Brookes and Primatesta 
2001) to oversample ethnic minorities. A limitation of the 
technique is that it will likely produce some (possibly 
substantial) undercoverage. Evidence of the extent of under- 
coverage can be obtained by comparing the prevalence of 
the rare population in the core sample with that in the linked 
addresses. 

In surveys that sample persons by first sampling house- 
holds, survey designers often prefer to select one person per 
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household — perhaps allowing two persons to be sampled in 
large households—to avoid contamination effects and 
prevent a within-household clustering homogeneity effect 
on design effects. This design is not always the best (Clark 
and Steel 2007), and this particularly applies when rare 
populations are sampled. When rare population members 
are concentrated in certain households (e.g., minority 
populations), the size of the screening sample can be 
appreciably reduced if more than one person -— even all 
eligible persons—can be taken in some households (see 
Hedges 1973). Elliott, Finch, Klein, Ma, Do, Beckett, Orr 
and Lurie (2008) suggest that, for oversampling American 
Indian/Alaskan Native and Chinese minorities in the United 
States, taking all eligible persons in a household has 
potential for U.S. health surveys. The NHANES maximizes 
the number of sampled persons per household. Since each 
respondent is remunerated for participation, households 
with more respondents receive more remuneration, a factor 
thought to increase response rates (Mohadjer and Curtin 
2008). Note that within-household homogeneity will have 
little effect on design effects when the data are analyzed by 
subgroup characteristics (e.g., age and sex) that cut across 
households. 

The use of large-scale screening to identify rare popu- 
lations raises three issues, each of which could lead to a 
failure to achieve planned sample sizes unless precautions 
are taken. The first results from the fact that, with screening, 
the sample size for a rare population is a random variable. 
As a result, the achieved sample size may be larger or 
smaller than expected. When a minimum sample size is 
specified for a rare population, it may be wise to determine 
the sampling fraction to be used to ensure that there is, say, 
a 90 percent probability that the achieved sample size will 
be at least as large as the specified minimum. This 
procedure was used in determining the sampling fractions 
for the many age, sex and income subdomains for the 
Continuing Survey of Food Intakes by Individuals 1994-96 
(Goldman, Borrud and Berlin 1997). 

The second issue raised by large-scale screening is that 
the overall nonresponse rate must be considered. A sampled 
member of a rare population will be a nonrespondent if the 
screener information is not obtained, or if a member of the 
rare population is identified (perhaps by a proxy informant) 
but does not respond to the survey items. The overall 
nonresponse rate may well be much higher than would 
occur without the screening component. Furthermore, the 
survey designers must consider the nature of the rare 
domain and the ways in which members of that domain will 
react to the survey content. A survey in which new 
immigrants are asked about their immigration experiences 
might have a very different response rate than a survey in 
which war veterans are asked about the medical and other 
support services they are receiving. 
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The third issue is that noncoverage can be a significant 
problem when large-scale screening is used to identify rare 
populations. One source of noncoverage relates to the 
sampling frame used for the screener sample. Even though a 
frame has good overall coverage, its coverage of a rare 
domain may be inadequate. For example, the noncoverage 
of a frame of landline telephone numbers is much higher for 
households of younger people than for the total population. 
The designers of landline telephone surveys of such rare 
domains as young children and college students therefore 
must carefully consider the potential for noncoverage 
biases. To address the problem of the substantial non- 
coverage of poor people in telephone surveys, the National 
Survey of America’s Families, which was designed to track 
the well-being of children and adults in response to welfare 
reforms, included an area sample of households without 
telephones in conjunction with the main random digit 
dialing (RDD) telephone sample (Waksberg, Brick, Shapiro, 
Flores Cervantes and Bell 1997). 

Another source of noncoverage is a failure to identify 
some members of the rare population at the screening stage. 
In particular, when a survey aims to collect data only for 
members of a rare domain, some screening phase re- 
spondents may falsely report, and some interviewers may 
falsely record, that the sampled persons are not members of 
that domain. These misclassifications may be inadvertent or 
they may be deliberately aimed at avoiding the second- 
phase data collection. Misclassification error can give rise to 
serious levels of noncoverage, particularly when the rare 
population classification is based on responses to several 
questions, misreports to any one of which leads to a 
misclassification (Sudman 1972, 1976). When the survey 
oversamples one or more rare domains as part of a survey of 
the general population, misclassifications are uncovered at 
the second phase, thus avoiding noncoverage. However, 
misclassifications still result in a smaller sample sizes for 
rare domains; in addition, the variation in sampling weights 
between respondents selected as members of the rare 
domain and those sampled as members of another domain 
can lead to a serious loss of precision. Noncoverage is more 
likely to arise when screener data are collected from proxy 
informants. It is a particular problem with focused 
enumeration. 

In a number of surveys of rare populations, the 
proportion of rare population members identified has been 
much lower than prevalence benchmarks. For example, the 
1994 NIS had an appreciable shortfall in the identified 
proportion of children aged 19 to 35 months (4.1 percent 
compared to the predicted rate of 5 percent) (Cambum and 
Wright 1996). In the National Longitudinal Survey of Youth 
of 1997, only 75 percent of youth aged 12 to 23 years were 
located (Horngan, Moore, Pedlow and Wolter 1999). These 
findings could be the result of higher nonresponse rates for 
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members of the rare population, frame noncoverage of 
various types, or misclassifications of domain membership. 
To produce the required sample size, an allowance for 
under-representation must be made at the design stage. The 
noncoverage of an age domain appears to be greatest at the 
domain boundaries, perhaps because respondents do not 
know exact ages (with those falsely screened out being lost 
and those falsely screened in being detected and dropped 


later) or because of deliberate misreporting to avoid the. 


follow-up interview. To counteract this effect, it can be 
useful to start with an initial screening for all household 
members or for a broader age range and then narrow down 
to the required age range later on. 

Weighting adjustments can be used in an attempt to 
mitigate biases caused by nonresponse and noncoverage, 
but they are necessarily imperfect. Adjustments for a 
domain specific level of nonresponse require knowledge of 
the domain membership of nonrespondents, but that is often 
not available. Adjustments for noncoverage of a rare 
domain require accurate external data for the domain, data 
that are often not available. Indeed, one of the purposes for 
some rare domain surveys is to estimate the domain size. 
Noncoverage is a major potential source of error in the 
estimation of domain size. 


3.2 Disproportionate stratification 


A natural extension of the screening approach is to try to 
identify strata where the screening will be more productive. 
In the ideal circumstance, one or more strata that cover all of 
the rare population and none from outside that population 
are identified. That case requires no screening process. 
Otherwise, it is necessary to select samples from all the 
strata (apart from those known to contain no rare population 
members) to have complete coverage of the rare population. 
The use of disproportionate stratification, with higher 
sampling fractions in the strata where the prevalence of the 
rare population is higher, can reduce the amount of 
screening needed. 


3.2.1 Theoretical background 


Consider initially a survey designed to provide estimates 
for a single rare population. Waksberg (1973) carried out an 
early theoretical assessment of the value of disproportionate 
stratification for this case. Subsequent papers on this topic 
include those by Kalton and Anderson (1986) and Kalton 
(1993a, 2003). The theoretical results show that three main 
factors must be considered in determining the effectiveness 
of disproportionate stratification for sampling a single rare 
population: the prevalence rate in each stratum, the 
proportion of the rare population in each stratum, and the 
ratio of the full cost of data collection for members of the 
rare population to the screening cost involved in identifying 
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members of that population. If it is assumed that (1) the 
element variances for the rare population are the same 
across strata and (2) the costs of data collection for members 
of the rare and non-rare populations are the same across 
strata, then, with simple random sampling within strata, the 
optimum sampling fraction in stratum / for minimizing the 
variance of an estimated mean for the rare population, 
subject to a fixed total budget, is given by 


| P, 
In Pp (cma 


where P, is the proportion of the units in stratum A that are 
members of the rare population and c is the ratio of the data 
collection cost for a sampled member of the rare population 
to the cost for a member of the non-rare population (Kalton 
1993a). The following formula provides the ratio of the 
variance of the sample mean with the optimum dispro- 
portionate stratified sampling fractions to that with a 
proportionate stratified sample of the same total cost: 


[=4,/P(e—1) + P7P, | 


PHN +1 


where A, is the proportion of the rare population in stratum 
h and P is the prevalence of the rare population in the full 
population. 

In general, the variability in the optimum sampling 
fractions across the strata, and the gains in precision for the 
sample mean, decline as c increases. Thus, if the main 
survey data collection cost is high —as, for instance, when 
the survey involves an expensive medical examination — or 
if the screening cost is very low, then disproportionate 
stratification may yield only minor gains in precision. 

When the main data collection cost adds nothing to the 
screening cost, the ratio of main data collection cost to 
screening cost will be c = 1. In this limiting situation, the 
formulas given above simplify to /, « vr) and R= 
(2,/4,W,)’, where W, is the proportion of the total 
population in stratum h. These simple formulas provide a 
useful indication of the maximum vanation in optimum 
sampling fractions and the maximum gains in precision that 
can be achieved. The square root function in the optimum 
sampling fraction formula makes clear that the prevalences 
in the strata must vary a good deal if the sampling fractions 
are to differ appreciably from a proportionate allocation. For 
example, even if the prevalence in stratum A is four times as 
large as that in stratum B, the optimum sampling fraction in 
stratum A is only twice as large as that in stratum B. The 
gains in precision (1 — R) are large when 4A, is large when 
W, is small and vice versa. With only two strata, a stratum 
with a prevalence five times as large as the overall 
prevalence (i.e., P,/P =5) will yield gains in precision of 
25 percent or more ((1 — R) 20.25) only if that stratum 
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includes at least 60 percent of the rare population (Kalton 
2003, Table 1). 

In summary, while generally useful, disproportionate 
stratification will yield substantial gains in efficiency only if 
three conditions hold: (1) the rare population must be much 
more prevalent in the oversampled strata; (2) the over- 
sampled strata must contain a high proportion of the rare 
population; and (3) the cost of the main data collection per 
sampled unit must not be high. In many cases, not all of 
these three conditions can be met, in which case the gains 
will be modest. 

Furthermore, the results presented above are based on the 
assumption that the true prevalence of the rare population in 
each stratum is known, whereas in practice it will be out of 
date (for example, based on the last census) or will perhaps 
simply have been guesstimated. Errors in the prevalence 
estimates will reduce the precision gains achieved with 
disproportionate stratification and could even result in a loss 
of precision. A major overestimation of the prevalence of 
the rare population, and hence of the optimum sampling 
fraction, in the high-density stratum can result in a serious 
loss of precision for the survey estimates. It is therefore 
often preferable to adopt a conservative strategy, that is, to 
adopt a somewhat less disproportionate allocation, one that 
moves in the direction of a proportionate allocation. 


3.2.2 Applications 


When area sampling is used, data available from the last 
census and other sources can be used to allocate the area 
clusters to strata based on their prevalence estimates for the 
rare population. See Waksberg, Judkins and Massey (1997) 
for a detailed investigation of this approach for over- 
sampling various racial/ethnic populations and the low- 
income population using U.S. census blocks and block 
groups as clusters. Based on data from the 1990 Census, 
Waksberg and his colleagues found that the approach 
generally worked well for Blacks and Hispanics but not for 
the low-income population. While the low-income popu- 
lation did exhibit high concentrations in some blocks and 
block groups, those areas did not cover a high proportion of 
that population. 

When the survey designers have access to a list frame 
with names, the names can be used to construct strata of 
likely members of some racial/ethnic groups. This situation 
arises, for instance, with lists of names and telephone 
numbers and when names are merged onto U.S. Postal 
Service (USPS) Delivery Sequence File addresses (no name 
merge is made in some cases). The allocation to strata can 
be based on surnames only or on a combination of sumame 
and first name (and even other names also). Since women 
often adopt their husbands’ surnames, the allocation is 
generally more effective for men than women. Names can 
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be reasonably effective for identifying Hispanics, Filipinos, 
Vietnamese, Japanese and Chinese, but not Blacks. A 
number of lists of names associated with different 
racial/ethnic groups have been compiled, such as the list of 
Spanish names compiled by the U.S. Census Bureau for the 
1990s (Word and Perkins 1996). Several commercial 
vendors have developed complex algorithms to perform 
racial/ethnic classifications based on names (see Fiscella 
and Fremont 2006 for further details). The use of names in 
identifying race and ethnicity has been of considerable 
interest to epidemiologists and demographers, who have 
conducted a number of evaluations of this method (e.g., 
Lauderdale and Kestenbaum 2000; Elliott, Morrison, 
Fremont, McCaffrey, Pantoja and Lurie 2009). They often 
assess the effectiveness of the method in terms of positive 
predictive value and sensitivity, which are the equivalents of 
prevalence and the proportion of members of the domain 
who are identified as such by the instrument used for the 
classification. In the sampling context, besides limitations in 
the instrument, researchers also need to take into account 
that sometimes names are not available and that some 
available names may be incorrect (for example, with 
address-based sampling, the names may be out-of-date, 
because the original family has moved out and a new family 
has moved into an address). These additional considerations 
serve to reduce the effectiveness of the name stratification, 
and depending on the particular circumstances, the 
reduction in effectiveness may be sizable. 

As with stratification in general, the stratification factors 
used for sampling rare populations do not have to be 
restricted to objective measures. They can equally be 
subjective classifications. The only consideration is how 
well they serve the needs of the stratification (See Kish 
1965b, pages 412-415, for an example of the effectiveness 
of the use of listers’ rapid classifications of dwellings into 
low, medium or high socio-economic status for dispropor- 
tionate stratification). Elliott, McCaffrey, Perlman, Marshall 
and Hambarsoomians (2009) describe an effective applica- 
tion of subjective stratification for sampling Cambodian 
immigrants in Long Beach, California. A local community 
expert rated all individual residences in sampled blocks as 
likely or unlikely to contain Cambodian households, based 
on externally observable cultural characteristics such as 
footwear outside the door and Buddhist altars. The 
residences allocated to the “likely” stratum (approximately 
20 percent) were then sampled at four times the rate than the 
rest. 

Sometimes, when the survey is concerned with pro- 
ducing estimates only for a very rare population, dispro- 
portionate stratification may still require an excessive 
amount of screening. In that circumstance, it may be 
necessary to sample from the strata where the prevalence is 
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highest, dropping the other strata and accepting some degree 
of noncoverage (or redefining the survey population to 
comprise only members of the rare population in the strata 
that were sampled). The Hispanic Health and Nutrition 
Examination Survey of 1982-84 (HHANES) provides an 
illustration. For its samples of Mexican Americans in the 
Southwest and Puerto Ricans in the New York City area, the 
HHANES sampled only from counties with large numbers 
and/or percentages of Hispanics, based on 1980 Census 
counts (Gonzalez, Ezzati, White, Massey, Lago and 
Waksberg 1985). 

As another example of this approach, Hedges (1979) 
describes a procedure for sampling a minority population 
that is more concentrated in some geographical districts, 
such as census enumeration districts. In this procedure, the 
districts are listed in order of their prevalence of members of 
the rare population (obtained, say, from the last census), and 
then the survey designers produce Lorenz curves of the 
cumulative distribution of rare population prevalence and 
the cumulative distribution of the proportions of rare popu- 
lation members covered. With the cumulative prevalence 
declining as the cumulative coverage increases, the survey 
designers can use these distributions to select the combi- 
nation of prevalence and proportion covered that best fulfills 
their requirements. The issue then to be faced is whether to 
make inferences to the covered population, or whether to 
make inferences to the full population by applying popu- 
lation weighting adjustments in an attempt to address the 
noncoverage bias. 

When a domain is very rare but a portion of it is heavily 
concentrated in a stratum, researchers sometimes sample 
that stratum at a rate much higher than the optimum in order 
to generate a sizable number of cases. Although this 
approach may produce a large sample of the rare 
population, the effective sample size (i.e., the sample size 
divided by the design effect) will be smaller than if the 
optimum sampling fractions had been used. Thus, from the 
perspective of the standard survey design-based mode of 
inference, this approach is not appropriate. However, the 
researchers using this approach often argue for a model- 
based mode of inference in which the sampling weights are 
ignored. In my view, ignoring the sampling weights is 
problematic. However, discussion of this issue is outside the 
scope of this paper. 


3.3 Two-phase sampling 


The screening approach treated in Sections 3.1 and 3.2 
assumes that identification of rare population members is 
relatively easy. When accurate identification is expensive, a 
two-phase design can be useful, starting with an imperfect 
screening classification at the first phase, to be followed up 
with accurate identification for a disproportionate stratified 
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subsample at the second phase. Whether the two-phase 
approach is cost-effective depends in part on the relative 
costs of the imperfect classification and accurate identifica- 
tion: since the imperfect classifications use up some of the 
study’s resources, they must be much less expensive than 
the accurate identification. Deming (1977) suggests that the 
ratio of the per-unit costs of the second- to the first-phase 
data collections should be at least 6:1. Also, the imperfect 
classification must be reasonably effective in order to gain 
major benefits from a second-phase disproportionate 
stratification. 

Two- or even three-phase sampling can often be useful in 
medical surveys of persons with specific health conditions. 
The first phase of the survey often consists of a screening 
questionnaire administered by survey interviewers, and the 
second phase is generally conducted by clinicians, often in a 
medical center. As one example, in a survey of epilepsy in 
Copiah County, Mississippi, Haerer, Anderson and 
Schoenberg (1986) first had survey interviewers administer 
to all households in the county a questionnaire that had been 
pretested to ensure that it had a high level of sensitivity for 
detecting persons with epilepsy. To avoid false negatives at 
this first phase, a broad screening net was used in 
identifying persons who would continue to the second 
phase. All those so identified were the subjects for the 
second phase of the survey, which consisted of brief 
neurological examinations conducted by a team of four 
senior neurologists in a public health clinic. 

A second example illustrates the use of another survey to 
serve as the first-phase data collection for studying a rare 
domain. In this case, the Health and Retirement Study 
(HRS) was used as the first phase for a study of dementia 
and other cognitive impairment in adults aged 70 or older. 
The HRS collects a wide range of measures on sample 
respondents, including a battery of cognitive measures. 
Using these measures, the HRS respondents were allocated 
to five cognitive strata, with a disproportionate stratified 
sample being selected for the second phase. The expensive 
second-phase data collection consisted of a 3- to 4-hour 
structured in-home assessment by a nurse and neuro- 
psychology technician. The results of the assessment were 
then evaluated by a geropsychiatrist, a neurologist and a 
cognitive neuroscientist to assign a preliminary diagnosis 
for cognitive status, which was then reassessed in the light 
of data in the person’s medical records (Langa, Plassman, 
Wallace, et al. 2005). 

A third example is a three-phase design that was used in 
a pilot study to identify persons who would qualify for 
disability benefits from the U.S. Social Security Admuinistra- 
tion if they were to apply for them (Maffeo, Frey and Kalton 
2000). At the first phase, a knowledgeable household 
respondent was asked to provide information about the 
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disability beneficiary status and impairment status of all 
adults aged 18 to 69 years in the household. At the second 
phase, all those classified into a stratum of severely disabled 
nonbeneficiaries and samples of the other strata were 
interviewed in person and were then reclassified as 
necessary into likely disability strata for the third phase. At 
the third phase, a disproportionate stratified sample of 
persons was selected to undergo medical examinations in 
mobile examination centers. 

A fairly common practice with two-phase designs is to 
take no second- (or third-) phase sample from the stratum of 
those classified as nonmembers of the rare domain based on 
their responses at the previous phase. The proportion of the 
population in that stratum is usually very high, and the 
prevalence of the rare domain in it is very low (indeed, as in 
the Haerer, Anderson and Schoenberg (1986) study, the 
stratum is often conservatively defined with the aim of 
avoiding the inclusion of those who might possibly be 
members of the rare domain). As a result, a moderate-sized 
sample from this stratum will yield almost no members of 
the rare domain. However, the cut-off strategy of taking no 
sample from this stratum is risky. If the prevalence of the 
rare domain in this large stratum is more than minimal, a 
substantial proportion of the domain may go unrepresented 
in the final sample. 


3.4 Multiple frames 


Sometimes sampling frames exist that are more targeted 
on a rare population than a general frame, but they cover 
only part of the rare population. In this situation, it can be 
efficient to select the sample from more than one frame. For 
example, in the common case of oversampling ethnic 
minorities, there is sometimes a list frame available. The 
persons on the list can be classified based on their names as 
being likely to belong to a given ethnic group (e.g., Chinese, 
Korean, Pacific Islanders, Vietnamese) to create a second, 
incomplete sampling frame from which to sample, in 
addition to a more complete frame that has a lower 
prevalence of the rare population (see, e.g., Elliott etal. 
2008; Flores Cervantes and Kalton 2008). As with 
disproportionate stratification (Section 3.2), major benefits 
derive from this approach only when the second frame has a 
high prevalence and covers a sizable fraction of the rare 
population. See Lohr (2009) for a review of the issues 
involved in sampling from multiple frames. 

With multiple frames, some members of the rare 
population may be included on several frames, in which 
case they may have multiple routes of being selected into 
the sample. There are three broad approaches for addressing 
these multiplicities (Anderson and Kalton 1990; Kalton and 
Anderson 1986). When all the frames are list frames, as 
sometimes occurs in health studies, it may be possible to 
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combine the frames into a single unduplicated list; however, 
this can often involve difficult record linkage problems. An 
alternative approach is to make the frames non-overlapping 
by using a unique identification rule that associates each 
member of the rare population with only one of the frames, 
treating the listings on the other frames as blanks (Kish 
1965b, pages 388-390). Samples are selected from each of 
the frames without regard to the duplication, but only the 
non-blank sampled listings are accepted for the final sample. 
This approach works best when searches can be made for 
each sampled unit on the other frames; if the frames are put 
in a priority order and the unit is found on a prior frame to 
the one from which the selection was made, the sampled 
listing would be treated as a blank. In this case, the frames 
are strata; the sampled units are treated as subclasses within 
the strata, allowing for the blank listings (Kish 1965b, pages 
132-139), and the analysis follows standard methods. 

The use of the unique identification approach can, 
however, be inefficient when the persons sampled from one 
frame have to be contacted to establish whether their listings 
are to be treated as real or blank for that frame. In this case, 
it is generally more economical to collect the survey data for 
all sampled persons (i.e., to accept the multiple routes of 
selection). There are, however, exceptions, as in the case of 
the National Survey of America’s Families. That survey 
used a combination of an area frame and an RDD telephone 
frame, with the area frame being used to cover only 
households without telephones (Waksberg, Brick et al. 
1997). It proved to be efficient to conduct a quick screening 
exercise with households on the area frame to eliminate 
households with telephones, retaining only the non- 
telephone households for the survey. 

There are two general approaches for taking multiple 
routes of selection into account in computing selection 
probabilities (Bankier 1986; Kalton and Anderson 1986). 
One method calculates each sampled unit’s overall selection 
probability across all the frames and uses the inverse of that 
probability as the base weight for the analysis (leading to the 
Horvitz-Thompson estimator). For example, the overall 
selection probability for sampled unit i on two frames is 
P) = (Put Poi - PuP2i) =U - 1 - py) CG - p2;)], where 
P, 38 the probability of the unit’s selection from frame 
f =1,2. A variant is to replace the overall selection 
probability with the expected number of selections (leading 
to the Hansen-Hurwitz estimator), which is easier to 
compute when multiple frames are involved. With only two 
frames, the expected number of selections is (p,, + P,;). 
When selection probabilities are small, there is little 
difference between these two estimators. 

Adjustments to compensate for nonresponse and to 
calibrate sample totals to known population totals can either 
be made to the overall selection probabilities p, or they can 
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be made to the p, individually. A problem that can occur 
is that the survey designers do not know whether a 
nonresponding unit sampled from one frame is on another 
frame since that information is only collected in the 
interview. In this situation the p, for nonresponding units 
cannot be directly computed and must be estimated in some 
fashion. When adjustments are made to the p, individ- 
ually, it is not possible to form nonresponse weighting 
classes that take membership on other frames into account. 
Instead, the designers must assume that, within weighting 
classes, the response rates are the same no matter how many 
frames a unit is on. 

In general, the application of the approach described 
above requires knowledge of each sampled unit’s selection 
probabilities for all of the frames, information that is not 
always available. When selection probabilities are not known 
for frames other than the frame(s) from which the unit is 
sampled (but presence/absence on the frames is known), an 
alternative approach, termed a weight share method by 
Lavallée (1995, 2007), can be used. Unbiased estimates of 
population totals are obtained if the weight for unit 7 is given 
by w, = 2,a,,w, where a, are any set of constants such 
that & a, =1 when summed across the / frames, 
w, =1/p, if unit iis selected from frame j with probability 
p,; and w,=0 otherwise (Kalton and Brick 1995; 
Lavallée 2007). For many applications, it is reasonable to set 
a, = a, and then a good choice of a, is a, =7,/2n,, 
where 77, is the effective sample size based on some average 
design effect (Chu, Brick and Kalton 1999). 

The second general approach for dealing with multiple 
routes of selection uses the multiple-frame methodology 
introduced by Hartley (1974), and the subject of much 
recent research (see, e.g., Lohr and Rao 2000 and 2006 and 
the references cited in those papers). In the case of two 
frames (A and B), the population can be divided into three 
mutually exclusive subsets labeled a= ANB, 
b = ACB and ab = ACB. The sample can be divided 
into samples from a, b and ab, where the ab sample can be 
separated into respondents sampled from frame A and those 
sampled from frame B. The samples in subsets a and b have 
only one route of selection, and hence are readily handled in 
estimation. Totals for ab could be estimated from the 
sample from frame A or the sample from frame B, say, i 
or Vee The Hartley methodology takes a weighted average 
of these two estimators, Y,, = OY + (1-0)¥2, where 0 
is chosen to minimize the variance of Y,,, taking into 
account that sample sizes and design effects differ between 
the two samples. Note that the dual-frame methodology is 
estimator specific, with different values of 0 for different 
estimators. Skinner (1991), Skinner and Rao (1996) and 
Lohr and Rao (2006) have proposed an alternative, pseudo- 
maximum likelihood estimation approach that has the 
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attraction of avoiding the problems associated with different 
values of @ for different variables. Wu and Rao (2009) 
propose a multiplicity-based pseudo empirical likelihood 
approach for multiple frame surveys, including what they 
term a single-frame multiplicity-based approach that 
incorporates Lavallée’s weight share method as described 
above. 

When a dual- or multiple-frame design is used, it is often 
the case that one frame has complete coverage but a low 
prevalence of the rare population (e.g., an area frame) and 
the other frame(s) has a much higher prevalence of the rare 
population but incomplete coverage. Metcalf and Scott 
(2009), for example, combined an area sample with an 
electoral roll sample for the Auckland Diabetes, Heart and 
Health Survey, in which Pacific Islanders, Maoris and older 
people were domains of special interest. The electoral roll 
frame had the advantage of containing information about 
electors’ ages, as well as a special roll on which those who 
considered themselves to be of Maori descent could enroll. 
Furthermore, many people of Pacific descent could likely be 
identified by their names, since Pacific languages use fewer 
letters than English. A disproportionate stratified sample 
was selected from the electoral roll frame to oversample the 
domains of interest, and the sample from the area frame 
brought in people not on the electoral rolls. 

The National Incidence Study of Child Abuse and 
Neglect provides an example of a more complex situation 
(Winglee, Park, Rust, Liu and Shapiro 2007). That survey 
used many frames to increase its overall coverage of abused 
and neglected children. Child Protective Services (CPS) 
agencies in the sampled PSUs were the basis of the main 
sampling frame, while police, hospitals, schools, shelters, 
daycare centers and other agencies were the sources of other 
frames. The samples from CPS agencies were selected from 
list frames, but the samples from other agencies were drawn 
by sampling agencies, constructing rosters of relevant 
professional staff, and sampling staff who acted as 
informants about maltreated children. With these 
procedures, duplication across agencies cannot be 
ascertained, except in the case of CPS agencies and any of 
the other agencies. The design was therefore treated as a 
dual-frame design, with CPS as one frame and the 
combination of the other frames as the second frame (Z.e., 
assuming no overlap between the other frames). 


3.5 Network sampling 


Network (or multiplicity) sampling expands on the 
standard screening approach by asking sampled persons (or 
addresses) to also serve as proxy informants to provide the 
screening information for persons who are linked to them in 
a clearly specified way (Sudman ef al. 1988; Sirken 2004, 
2005). Relatives such as parents, siblings and children are 
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often used as the basis of linkages. A key requirement is that 
every member of the linkage must know and be willing to 
report the rare population membership statuses of all those 
linked to them. In a pilot study of male Vietnam veterans, 
Rothbart, Fine and Sudman (1982) included aunts and 
uncles as informants as well as parents and siblings, but 
found that aunts and uncles identified far fewer Vietnam 
veterans than expected. This apparent failure of aunts and 
uncles to report some veterans gives rise to a potential 
sampling bias, thus making their inclusion in the linkage 
rules problematic. 

The multiple routes of selection with network sampling 
need to be taken into account in determining selection 
probabilities in a similar manner to that described for 
multiple frames in the previous section. Conceptually, one 
can consider each member of the rare population divided 
into, say, / parts corresponding to the / informants for that 
member; it is then these parts that are sampled for the survey. 
See Lavallée (2007) for some theory behind the technique. 

When network sampling is used in surveys that collect 


data on the characteristics of rare population members, ~ 


direct contact must be made with the members of the rare 
population identified by the initial informant. In this case, 
the informant has to be able to provide contact information 
for the rare population members. The linkage definition may 
be structured to facilitate the follow-up data collection. For 
example, with face-to-face interviewing, the linkage may be 
restricted to relatives living in a defined area close to the 
informant. 

Sudman and Freeman (1988) describe the application of 
network sampling in a telephone survey about access to 
health care, in which an oversample of persons with a 
chronic or serious illness was required. During an initial 
contact with the head of the household, linkages to the 
respondent’s or spouse’s parents, stepparents, siblings, 
grandparents and grandchildren under age 18 were 
identified and data were collected on their health status. The 
use of this network sampling design increased the number 
of chronically or seriously ill adults identified by about one- 
third. However, about one in eight of the initial network 
informants with relatives were unable or unwilling to 
provide illness information for their network members, and 
70 percent did not provide complete location information, 
including 28 percent who provided neither name nor 
location information (thus making tracing impossible). The 
use of network sampling led to some false positives 
(persons reported as being chronically or seriously ill by the 
initial respondent but reporting themselves as well). A more 
serious concern is that the survey was not able to provide 
information on false negatives (this would have required 
following up a sample of network members reported to be 
well by the initial informant). 
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Some forms of linkage have the added benefit that they 
can incorporate some rare population members who are not 
on the original sampling frame and would therefore 
otherwise be a component of noncoverage. For example, 
Brick (1990) describes a field test for the telephone-based 
National Household Education Survey (NHES) that used 
multiplicity sampling to increase the sample of 14- to 21- 
year-olds, with a focus on school drop-outs. In a subsample 
of households, all women aged 28 to 65 were asked to 
provide information for all their 14- to 21-year-old children 
currently living elsewhere. Some of these children lived in 
telephone households and hence had two routes of selection. 
Others lived in non-telephone households and hence would 
not have been covered by the survey; their inclusion via the 
multiplicity design increased the coverage rate in 1989 by 
about 5 percent. However, the response rate for out-of- 
household youth was much lower than that for in-household 
youth because of failure to reach the youth, particularly the 
youth living in non-telephone households. 

Tortora, Groves and Peytcheva (2008) provide another 
example, in this case using multiplicity sampling in an 
attempt to cover persons with only mobile telephones via an 
RDD sample of landline telephone numbers. Respondents 
to the RDD survey (itself a panel survey) were asked to 
provide information about parents, siblings and adult 
children living in mobile-only households. The results 
demonstrate some of the general issues with multiplicity 
sampling: knowledge about the mobile-only status of the 
network members depended on the cohesion of the network; 
there was widespread unwillingness to provide mobile 
telephone numbers; and many of those identified as mobile- 
only households in fact also had a landline telephone. 

Network sampling has not been widely used in practice 
for surveys of rare population members. Some of the 
limitations of the method are illustrated by the studies 
described above. There is the msk that the sampled 
informant may not accurately report the rare population 
status of other members of the linkage, either deliberately or 
through lack of knowledge. Nonresponse for the main 
survey data collection is another concern. In addition, 
ethical issues can arise when sampled persons are asked 
about the rare population membership of those in their 
linkage when that membership is a sensitive matter. The 
benefits of network sampling are partially offset by the 
increased sampling errors arising from the variable weights 
that the method entails, and by the costs of locating the 
linked rare population members. 


3.6 Location sampling 


Location sampling is widely used to sample populations 
that have no fixed abode for both censuses and surveys: 
nomads may, for example, be sampled at waterpoints when 
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they take their animals for water, and homeless persons may 
be sampled at soup kitchens when they go for food (e.g., 
Kalton 1993a; Ardilly and Le Blanc 2001). A central feature 
of such uses of location sampling is that there is a time 
period involved, resulting in issues of multiplicity (Kalsbeek 
2003). A serious concern with the use of the technique is 
that it fails to cover those who do not visit any of the 
specified locations in the particular time period. 

Location sampling is used to sample rare mobile popu- 
lations such as passengers at airports and visitors to a 
museum or national park. In such cases, the question arises 
as to whether the unit of analysis should be the visit or the 
visitor. When the visit is the appropriate unit, no issues of 
multiplicity arise (see, for example, the report on the U.S. 
National Hospital Discharge Survey by DeFrances, Lucas, 
Buie and Golosinskiy 2008). However, when the visitor is 
the unit of analysis, the fact that visitors may make multiple 
visits during the given time period must be taken into 
account (Kalton 1991; Sudman and Kalton 1986). One 
approach is to treat visits as eligible only if they are the first 
visits made during the time period for the survey. Another 
approach is to make multiplicity adjustments to the weights 
in the analysis; however, determining the number of visits 
made is problematic because some visits will occur after the 
sampled visit. 

Location sampling has also been used for sampling a 
variety of rare—often very rare — populations that tend to 
congregate in certain places. For example, Kanouse, Berry 
and Duan (1999) employed the technique to sample street 
prostitutes in Los Angeles County by sampling locations 
where street prostitution was known to occur, and by 
sampling time periods (days and shifts within days). Loca- 
tion (center) sampling has also been used to sample legal 
and illegal immigrants in Italy (Meccati 2004). For a 2002 
survey of the immigrant population of Milan, 13 types of 
centers were identified, ranging from centers that provide 
partial lists from administrative sources (e.g., legal and work 
centers, language courses), centers that have counts of those 
attending (e.g., welfare service centers, cultural asso- 
ciations), to centers with no frame information (e.g., malls, 
ethnic shops). 

Location sampling has often been used to sample men 
who have sex with men, with the locations being venues 
that such men frequent, such as gay bars, bathhouses and 
bookstores (Kalton 1993b, MacKellar, Valleroy, Karon, 
Lemp and Janssen 1996). Based on a cross-sectional 
telephone survey, Xia, Tholandi, Osmond, Pollack, Zhou, 
Ruiz and Catania (2006) found that men who visited gay 
venues more frequently had higher rates of high-risk sexual 
behaviors and also that the rates of high-risk behaviors 
varied by venue. These findings draw attention to the 
difficulty of generating a representative sample by location 
sampling. 
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McKenzie and Mistiaen (2009) carried out an experiment 
to compare location (intercept) sampling with both area 
sampling and snowball techniques, for sampling Brazilians 
of Japanese descent (Nikkei) in Sao Paulo and Parana. The 
locations included places where the Nikkei often went (e.g., 
a sports club, a metro station, grocery stores and a Japanese 
cultural club) and events (e.g., a Japanese film and a 
Japanese food festival). Based on this experiment, they 
conclude that location sampling (and snowball sampling) 
oversampled persons more closely connected with the 
Nikkei community and thus did not produce representative 
samples. This not-unexpected finding highlights the concern 
about the use of location sampling for sampling rare 
populations in general, although not for sampling visits to 
specified sites. 


3.7 Accumulating or retaining samples over time 


When survey data collection is repeated over time, 
survey designers can take advantage of that feature in 
sampling rare populations (Kish 1999). An important 
distinction to be made is that between repeated and panel 
surveys. Samples of rare population members can readily be 
accumulated over time in repeated surveys. For example, 
the U.S. National Health Interview Survey is conducted on a 
weekly basis with nationally representative samples; sam- 
ples of rare populations can be accumulated over one or 
more years until a sufficient sample size is achieved (U.S. 
National Center for Health Statistics 2009a). With accumu- 
lation over time, the estimates produced are period, rather 
than point-in-time, estimates that can be difficult to interpret 
when the characteristics of analytic interest vary markedly 
over time (Citro and Kalton 2007). For example, how is a 3- 
year period poverty rate for a rare minority population to be 
interpreted when the poverty rate has varied a great deal 
over the period? 

In considering the sampling of rare populations in panel 
surveys, it is important to distinguish between rare popu- 
lations that are defined by static versus non-static charac- 
teristics. No accumulation over time can be achieved in 
panel surveys for rare populations defined by static charac- 
teristics such a race/ethnicity. However, if a sample of a 
static rare population is taken at one point in time, it can be 
useful to follow that sample in a panel to study that 
population’s characteristics at later time points, possibly 
with supplementary samples added to represent those who 
entered that population after the onginal sample was 
selected. Fecso, Baskin, Chu, Gray, Kalton and Phelps 
(2007) describe how this approach has been applied in 
sampling U.S. scientists and engineers over a decade. For 
the decade of the 1990s, the National Survey of College 
Graduates (NSCG) was conducted in 1993 with a stratified 
sample of college graduates selected from the 1990 Census 
of Population long-form sample records. Those found to be 
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scientists or engineers were then resurveyed in the NSCG in 
1995, 1997 and 1999. To represent new entrants to the 
target population, another survey —the Survey of Recent 
College Graduates — was conducted in the same years as the 
NSCG. A subsample of the recent college graduates was 
added in to the next round of the NSCG panel on each 
occasion. 

Panel surveys can be used to accumulate samples of non- 
static rare populations, especially persons experiencing an 
event such as a birth or a divorce. The U.S. National 
Children’s Study, for instance, plans to follow a large 
sample of eligible women of child-bearing age over a period 
of about four years, enrolling those who become pregnant in 
the main study, a longitudinal study that will follow the 
children through to age 21 (National Children’s Study 2007, 
Michael and O’ Muircheartaigh 2008). 

Finally, a large sample can be recruited into a panel and 
provide data that will identify members of a variety of rare 
populations that may be of future interest. They are then 
followed in the panel and, based on their rare population 
memberships, included in the samples for the surveys for 
which they qualify. Komer and Nimmergut (2004) describe 
a German “access panel” that could be used in this way, and 
there are now several probability-based Web panels that can 
serve this purpose (Callegaro and DiSogra 2008). However, 
a serious concer with such panels is the low response rates 
that are generally achieved. 


4. Concluding remarks 


This paper has presented a brief overview of the range of 
methods used in sample surveys for sampling and 
oversampling rare populations, primarily those classified by 
Kish as minor domains (the references cited provide more 
details). Although the methods have been discussed 
separately, in practice they are often combined, particularly 
when there are several rare domains of interest. As an 
example, the California Health Interview Survey, conducted 
by telephone, has used a combination of disproportionate 
stratification (oversampling telephone exchanges where the 
prevalence of the Korean and Vietnamese populations of 
interest is higher) and a dual-frame design (RDD methods 
supplemented with a frame of likely Korean and 
Vietnamese names). In many cases, the art of constructing 
an effective probability sample design for a rare population 
is to apply some combination of methods in a creative 
fashion. 

As another example, the Pew Research Center telephone 
survey of Muslim Americans employed three sampling 
methods to sample this very rare population (Pew Research 
Center 2007). One component of the design was a geograph- 
ically stratified RDD sample, with disproportionate stratified 
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sampling from strata defined in terms of the prevalence of 
Muslim Americans. The stratum with the lowest prevalence 
was treated as a cut-off stratum and excluded. The second 
component was a recontact sample of Muslim Americans 
drawn from Pew’s interview database of recent surveys. The 
third component was an RDD sample selected from a list of 
likely Muslim Americans provided by a commercial 
vendor. To avoid duplicate routes of selection between the 
geographical strata and the commercial vendor list, 
telephone numbers selected from the geographical strata 
were matched against the commercial vendor list and 
dropped from the geographical strata sample if a match was 
found. 

Not only are the various sampling techniques often used 
in combination in sample designs for rare populations, but 
several of the techniques are interrelated. For example, 
multiple frames can be treated by unique identification (see 
Section 3.4), which in effect is simply disproportionate 
stratification. Whereas the whole population is classified 
into strata for disproportionate stratification, the same 
approach is adopted with two-phase sampling, but the 
classification into strata is applied only to members of the 
first-phase sample. The theory of network sampling is 
similar to that of multiple-frame sampling, when the latter 
technique uses inverse overall selection probabilities as 
weights in the analysis. These interrelationships help to 
explain the similarities in the theoretical underpinnings of 
the techniques. 
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A standardization of randomized response strategies 


Andreas Quatember ' 


Abstract 


Randomized response strategies, which have originally been developed as statistical methods to reduce nonresponse as well 
as untruthful answering, can also be applied in the field of statistical disclosure control for public use microdata files. In this 
paper a standardization of randomized response techniques for the estimation of proportions of identifying or sensitive 
attributes is presented. The statistical properties of the standardized estimator are derived for general probability sampling. 
In order to analyse the effect of different choices of the method’s implicit “design parameters” on the performance of the 
estimator we have to include measures of privacy protection in our considerations. These yield variance-optimum design 
parameters given a certain level of privacy protection. To this end the variables have to be classified into different categories 
of sensitivity. A real-data example applies the technique in a survey on academic cheating behaviour. 


Key Words: Privacy protection; Statistical disclosure control; Nonresponse; Untruthful answering. 


1. Introduction 


The occurence of nonresponse and the unwillingness to 
provide the true answers are natural in survey sampling. 
They may result in an estimator of population parameters, 
which has a bias of unknown magnitude and a high 
variance. A responsible user therefore cannot ignore the 
presence of nonresponse and untruthful answering. 

Let U be the universe of N population units and U, be 
a subset of N, elements, that belong to a class A of a 
categorial variable under study. Moreover let U‘, be the 
group of N% elements, that do not belong to this class 
ee a Uh lng) Un Oe Noe Niet IV a det 


Pit sanity eU 
0 otherwise 


(i =1, 2,..., NM) and the parameter of interest be the 
relative size m, of subpopulation U ,: 


hb Paane “y Ny 
Gale - 


M4 


(Xv x, is abbreviated notation for }-y x;). In a probability 
sample s (see for instance: Sarmdal, Swensson and 
Wretman 1992, page 8f) an estimator of m, can be 
calculated from the Horvitz-Thompson estimator of NV, by 


Xx; 
Oy = (2) 


(x, > 0 is the probability that unit 7 will be included in the 
sample), if the question “Are you a member of group U ,?” 
(or an equivalent question) is asked directly (dir). This 
estimator is unbiased, if all x,’s (i=1, 2,..., ”) are 


jane = nthe 
N 


observed truthfully. In the presence of unit or item 
nonresponse with respect to a variable under study the 
sample s is divided into a “response set” r C's of size n, 
and a “missing set’? mcs of size n,(s=rum, 
ram=f,n=n,+n,). For variables of a_ highly 
personal, embarrassing matter (like drug addiction, diseases, 
sexual behaviour, tax evasion, alcoholism, domestic 
violence or involvement in crimes) r is furthermore 
divided into a set ¢ of n, sample units, who answer 
truthfully, and a set uw of size n,, who answer untruthfully 
(r=tUu,tanu=6,n = n, + n,,). Estimator (2) must 
then be rewritten as: 

» dir l x; x; Mi 

=o (DA+DA+y4} © 


aC. 


1 


r 


Evidently the elements of set u cannot be identified and 
the x,’s of m are not observable. This imposes errors of 
measurement and nonreponse on the estimation. Therefore 
everything should be done to keep the untruthful answering 
rate as well as the nonresponse rate as low as possible. 

Survey design features, which clearly affect both the 
quantity and the quality of the information asked from the 
respondents (see for instance: Groves, Fowler, Couper, 
Lepkowski, Singer and Tourangeau 2004, Section 6.7), are 
strongly related to the sample units’ concerns about “data 
confidentiality” and “perceived protection of privacy”. The 
first term refers to the respondents’ desire to keep replies out 
of hands of uninvolved persons, whereas the second refers 
to the wish to withhold information from absolutely 
anybody. Singer, Mathiowetz and Couper (1993) and 
Singer, van Hoewyk and Neugebauer (2003) report on two 
successive U.S. population surveys, that the higher these 
concerns are the lower is the probability of the respondent’s 
participation in the survey (page 470ff and page 375ff). 


1. Andreas Quatember is Assistant Professor at the IFAS-Department of Applied Statistics, Johannes Kepler University Linz, Altenberger Str. 69, A-4040 
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What can statisticians contribute to this important field of 
research? For awkward questions the use of randomized 
response Strategies at the survey’s design stage may reduce 
the rates of nonresponse and of untruthful answering due to 
a perceived increase of privacy protection. A common 
characteristic of these methods is that instead of the direct 
questioning on the sensitive subject a questioning design is 
used, which does not enable the data collector to identify the 
(randomly selected) question on which the respondent has 
given the answer, although it does still allow to estimate the 
parameter under study. The idea is to reduce in this way the 
individuals’ fear of an embarrassing “outing” to make sure 
that the responding person is willing to cooperate. To 
achieve this goal the respondent clearly has to understand 
how the questioning design does protect his or her privacy 
(cf. Landsheer, van der Heijden and van Gils 1999, page 
6ff). 

Pioneering work in this field was published by Warner 
(1965). In his questioning design each respondent has to 
answer randomly either with probability p, the question 
“Are you a member of group U,?” or with probability 
P, =1- p, the alternative “Are you a member of group 
U4?” (0< p, <1). Since then various randomized 
response techniques with differing randomization devices 
have been proposed (for a review see: Chaudhun and 
Mukerjee 1987, Nathan 1988 or Tracy and Mangat 1996). 
All of these strategies make use of randomly selected 
questions or answers, though some of them use different 
random devices depending on the respondent’s possession 
or nonpossession of a certain attribute (see for example: 
Kuk 1990; Mangat 1994; Kim and Warde 2005). 

Warmer (1971) was the first to note that these techniques 
are also applicable as methods of masking confidential 
micro-data sets to allow their release for public use (cf. ibd., 
page 887). Such microdata sets might contain variables, 
which allow the direct identification of survey units like the 
name or an identification number, but also variables, which 
contain sensitive information on an individual. To protect 
the survey units against disclosure it might not suffice to 
delete the variables, which are directly linked to entities, 
because some of the units might still be identifiable by the 
rest of their records. Statistical disclosure control is nothing 
else but a balancing act between the protection of the 
anonymity of the survey units and the preservation of 
information contained in the data (cf Skinner, Marsh, 
Openshaw and Wymer 1994). Methods of data masking can 
be classified into three categories (cf Domingo-Ferrer and 
Mateo-Sanz 2002 or Winkler 2004): (1) The global 
recoding of variables into less detailed categories or larger 
intervals (see for instance: Willenborg and de Waal 1996, 
page Sf) or the /ocal recoding using different grouping 
schemes at unit level (cf Hua and Pei 2008, page 215f). (2) 
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The local suppression of certain variables for survey units 
with a high risk of re-identification by simply setting their 
values at “missing” (cf, Willenborg and de Waal 1996, page 
77). (3) The substitution of true values of a variable by other 
values. 

One of the strategies of the third category is the micro- 
aggregation of variables (cf: Defays and Anwar 1998). 
Therein the true variable values are for example sorted by 
size and then divided into (small) groups. Within each group 
data aggregates are released instead of the omnginal 
observations. Another such method is data-swapping, where 
data from units with a high risk of re-identification are 
interchanged with data from another subset of survey units 
(cf. Dalenius and Reiss 1982). Another technique of 
substituting identifying or sensitive information is the 
addition of noise to the observed values, meaning that the 
outcome of a random experiment is added to each datum (cf. 
Dalenius 1977 or Fuller 1993). Finally also the randomized 
response techniques can be used to mask identifying or 
sensitive variables. In this case either the survey units 
already perform the data masking at the survey’s design 
stage or the statistical agency applies the probability 
mechanism of the technique before the release of the 
microdata file (cf Rosenberg 1980, Kim 1987, Gouweleeuw, 
Kooiman, Willenborg and de Wolf 1998, or van den Hout 
and van der Heijden 2002). 

All methods of statistical disclosure control protect the 
survey units’ privacy by a loss of information, which can be 
seen as the price that has to be paid for it. To be able to 
appropriately adjust the estimation process the user of the 
microdata file has to be informed about the details of the 
masking procedure. 

A new standardization of the techniques of randomized 
response follows in Section 2 of this paper. Furthermore the 
statistical properties of the standardized estimator are derived 
for general probability sampling. In Section 3 the essential 
perspective of privacy protection is described. The question, 
which of the special cases included in the standardization is 
most efficient, is answered in the subsequent Section 4. 
Section 5 contains a real-data example, which demonstrates 
the application of the recommendations of Section 4 in a 
survey on academic cheating behaviour. 


2. Standardizing randomized response strategies 


Let us formulate the following standardization of the 
randomized response strategies: Each respondent has either 
to answer randomly with probability 


—  p, the question “Are you a member of group U ,?”, 
—  p, the question “Are you a member of group U{?” 
or 
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—  p, the question “Are you a member of group U,?” 
or is instructed just to say 

— “yes” with probability p, or 

— “no” with probability p, 


OF1 pies; Oe pi Sitatior ai Wy 2nd). bTheriVy, 
elements of group U, are characterized by the possession 
of a completely innocuous attribute B (for instance a 
season 8B of birth), that should not be related to the 
possession or nonpossession of attribute 4. This 
nonsensitive question on membership of group U, was 
introduced as an alternative to the question on membership 
of U, by Horvitz, Shah and Simmons (1967) to further 
reduce the respondent’s perception of the sensitivity of the 
procedure. m, = N,/N (with 0 < m, < 1) is the relative 
size of group U,.n, and the probabilities p,, p,, ..., Ds 
are the design parameters of our standardized randomized 
response technique. 
Let 


if unit i answers “yes”, 


1 
KS 
0 otherwise 


(i = 1, 2, ..., m). For an element i the probability of a 
“yes”-answer with respect to the randomized response 
questioning design R is for given x: 
Ply sol) ScDi 2 Xj te Powe (aX) 
ADs Wopehhp a= ary yb (4) 
Will a= p,— p, and b= "p, F p,-1, + pre Then the 
term 


i 


z vi par b 
a 
is unbiased for the true value x,(a # 0). Using these 


“substitutes” for x, (and assuming full cooperation of the 
respondents) the following theorems apply: 


Theorem 1: For a probability sampling design with 
inclusion probabilities 1, the following unbiased estimator 
of parameter 7, 1s given: 


el iin 
ft, Sera wb: (5) 


Theorem 2: For a probability sampling design P the 
variance of the standardized estimator 7, (5) is given by 


1 Ae pe (tp 1 
raande(ofeg)oetet nd 


N i 


fis Dobie 
Se SI (6) 


a I 


For the proofs of both theorems see the Appendix. The 
first summand within the outer brackets of (6) refers to the 
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variance of the Horvitz-Thompson estimator for the total 
vx; for a probability sampling design P when the 
question on membership of U, is asked directly. The. 
second one can be seen as the price we have to pay in terms 
of accuracy for the privacy protection offered by the 
randomized response questioning design. Apparently this 
variance can be estimated unbiasedly by inserting an 
unbiased estimator V,(X,x,/m,) for V,(X,x,/n,) and 
Seyi 1, for Sy xt 7. 

For simple random sampling without replacement for 
instance estimator (5) is given by 


7 aa (7) 


with %, = 2, y,/n, the proportion of “yes’-answers in the 
sample. In this case the variance (6) of the standardized 
estimator ft, is given by 
je (AS i a ia) 
n N=1 
1 {b-GQ-b) 1-2-b-a 
eal Ds dadit a) 


a a 


V(t) = 


n 


This theoretical variance is unbiasedly estimated by 


ae t,:-d-t N-n 
Pe.) = 4 La) 2 
ee EE 
n a a 


To be able to calculate 7, at all, the question on 
membership of U, (or U%, but we will ignore this 
possibility subsequently without loss of generality) must be 
included in the questioning design with p, > 0. There is a 
total of 16 combinations of this question with the four other 
questions or answers (see: Table 1). These combinations can 
be described as special cases of our standardized response 
strategy. For example choosing p, = 1 leads to the direct 
questioning on the subject. If we let 0 < p, <1 and 
P, =1- p, the standardized questioning design turns into 
Warner’s procedure. For 0 < p, <1 and p, =1- p, one 
gets Horvitz etal.’s technique with known 7, (see: 
Greenberg, Abul-Ela, Simmons and Horvitz 1969). (For 
other special cases already published as to the best of our 
knowledge, the reader is referred to the “References”- 
column of Table 1). 

The question, that arises directly from these consider- 
ations, is how to choose the design parameters of the 
standardized response technique to find out the strategies 
that perform best. We will answer this question in Section 4. 
But for this purpose we have to include the level of privacy 
protection, which results from choosing these parameters 
differently, in our considerations. 
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Table 1 
All special cases of the standardized randomized response 
strategy 


Design Questions/Answers References 
U, U A U,p yes no 
STI ° Direct questioning 
ST2ainis ° Wamer (1965)' 
STAs cs « Greenberg et al. (1969) 
ST4 ° ° 
STS e e 
ST6 e e e 
ASS e e e 
ST8 ® © Quatember (2007) 
ST9 e e e 
ST1I0. e e Singh, Hom, Singh and Mangat (2003) 
STlI1 ee Fidler and Kleinknecht (1977)° 
ST12 e e e e 
Ay filles e e cy e 
ST14 e e e e 
ST15 e e e e 
ST16 e e r) e r) 


1. A two-stage version was presented by Mangat and Singh (1990) 

2. A two-stage version was presented by Mangat (1992) 

3. This is a one-stage version of Mangat, Singh and Singh (1993) 

4. This is a one-stage version of Singh, Singh, Mangat and Tracy 
(1994) 

5. A two-stage version was presented by Singh, Singh, Mangat and 
Tracy (1995) 


3. Privacy protection 


To be able to compare the efficiency of questioning 
designs with different design parameters it is apparently 
inevitable to measure the loss of the respondents’ privacy 
induced by these parameters. The following ratios A, and 
i, of conditional probabilities may be used for this purpose 
(cf. for example the similar “measures of jeopardy” in 
Leysieffer and Warner 1976, page 650): 

wie max[P(y,= j |ieU,), PCy,= J |i e UZ) (10) 

; min[P(y,;= j |i €U,), PQ, = 7 |b eU;)I 
(<2, S07 =1, 0). 

For j =1 (10) refers to the privacy protection with 
respect to a “yes”-, for 7 = 0 with respect to a “no”- 
answer. For the standardized questioning design these “A - 
measures” of loss of privacy are given by 

max[a + b; 5] 


sneer 11 
' minfa + b; 6] ie) 


and 


ane neh ak shale sara ly (12) 
min{l — (a + b); 1-5] 

X, = A, =1 indicates a totally protected privacy. This 
means that the answer of the responding unit contains 
absolutely no information on the subject under study. This 
applies for a = 0. The more the  -measures differ from 


Statistics Canada, Catalogue No. 12-001-X 


unity, the more information about the characteristic under 
study is contained in the answer on the record. At the same 
time the efficiency of the estimation increases (see below), 
but the individual’s protection against the data collector 
decreases. For the direct questioning design with p, = 1, 
where no masking of the variable is done at all, these 
measures are given by 2, = A, = ©. 

Let the values A, ,,, and Ay. be the maximum A) - 
values of (11) and (12), that the agency considers to allow 
enough disclosure protection for the records. In the case of 
the strategy’s usage as to avoid nonresponse and untruthful 
answering in surveys we may also model the respondents’ 
willingness to cooperate as a function of perceived privacy 
protection. If the privacy of the respondents is sufficiently 
protected by the randomization device their full cooperation 
is assumed. Exceeding the limits A, ,,, and/or A, ,,, would 
then automatically introduce untruthful answering and 
nonresponse into the survey and therefore set us back to the 
starting point of the problem. Fidler and Kleinknecht (1977) 
showed in their study for design ST11 (Table 1) containing 
nine variables of very different levels of sensitivity, that 
their choice of the design parameters (p, = 10/16, p, = 
Pp; = 3/16) yielded nearly full and truthful response for 
each variable including sexual behaviour (ibd., page 1048). 
Inserting these values in (11) and (12) gives A, =A, = 
13/3. This finding corresponds in the main with results that 
can be derived from the experiment by Soeken and 
Macready (1982) and with recommendations given by 
Greenberg et al. (1969). Therefore choosing A, and/or 
Ao, opt Close to a value of 4 could be a good choice for most 
variables, when the standardized randomized response 
method is used to avoid refusals and untruthful answering of 
respondents in a survey. 

Without loss of generality let us assume subsequently, 
that we will choose the two categories of the variable under 
study in such way, that the membership of U,, is at least as 
sensitive as the membership of U4 (1 S Ay ot S Ao.op S 
oo). From (11) and (12) the terms a and b can be 
expressed by the A -values A, and A,. Their sum is given 
by: 


pee 
Py es (13) 
te ] 
i oad rs 
with 
ee 
b=— - (14) 
] 
hie: 
Ay > Ag 
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and 


r 
ia ie (15) 


We keep the double ratios on the right of (14) and (15) 
to find easily the limits for A, ~ «© and A, > © 
respectively. 

This means that for a given sampling design P the 
extent of the term (b-(1 — b)/a’)-Yy(I/n,)+(1 -2-b - 
ala): Xy(x,;/7m,) im the variance expression (6) does not 
depend on a single value of the design parameters, but on 
their aggregated effect on the loss of privacy measured by 
dX, and A,. Questioning designs with the same A -values 
are equally efficient. Designs with larger A, and/or A, are 
less efficient than designs with lower i ’s. 


4. Optimum questioning designs 


It does depend on the type of re-identification nsk or 
sensitivity of the subject under study which of the special 
cases of the standardized randomized response strategy of 
Table 1 can be most efficient for given A -measures. 
Strategies ST5 and S78 can never perform best, because 
they do always protect a “no”-answer more than a “‘yes”. 

For a nonidentifying (or nonsensitive) variable (like for 
instance the season of birth), where A, ok = Ao, on 
applies, only the direct questioning design (.ST1 of Table 1) 
can achieve the variance-optimum performance (see Table 
2, which shows these values of the design parameters, 
which guarantee the best performance of the estimator 7 ,; 
to be able to use Table 2 properly the categorial variable 
under study has to be classified according to the following 
categories: C,: The variable is not sensitive at all (A 
do. opt = ©); Cy: Only the membership of group U, 1s 
sensitive, but not of U4(Ay or < Ao opt = ©); C3: The 
membership of both groups U, and U‘% is sensitive, but 
not equally (A, 4 < Ao, op < ©); C,: The membership of 
U, and of U{, is equally sensitive (A, 91 = o,op < ©): 
which shows these values of the design parameters, which 
guarantee the best performance of the estimator 7,). 
Although the other designs can be used for such variables, 
they do unnecessarily protect the privacy of the respondents 
in some way. This has to be paid by a loss of accuracy of the 
estimation of m,. But for p, = 1(a=1 and b=0) the 
variance of 7%, (5) turns to the common formula of the 
direct questioning with the assumption of full response: 
VEC yi vs PaO. XT; )- 


= 0 


1, opt +a 
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For a variable, of which only the membership of U,, but 
not of U% is sensitive (for instance: U, = set of drug users 
within the last year, U, = U—U,,) there is ay oe < Ao, opt = 
oo. Calculating (14) and (15) for 1<A,<o and 
Ay > © gives a = 1—b and inserting this into (6) leads 
to the following expression for the variance of the estimator: 


Vp(%,) = 
Aird vit oa Wao atl ee bce 
yal Abed ore [z,2 Sv2)| (16) 


Looking for those values of the design parameters, for 
which the standardized randomized response strategy can 
achieve this variance and for which equations (14) to (15) 
hold, we do find that in this case there is only one solution! 
The only questioning design, that is able to perform 
optimally, is S74. Its variance-optimum design parameters 
are given by p, = (A, — 1)/A, and p, = 1- p, (see Table 
2). This means, that with probability p, = (A, —1)/A, a 
respondent is asked the question on membership of U, and 
with the remaining probability he or she is instructed to say 
“yes”. In this way the data collector is only able to conclude 
from a “no”-answer directly on the nonsensitive non- 
possession of A but not from a “yes”-answer on the 
possession of this sensitive or identifying attribute. 

Questioning design S71 is not applicable for such 
subjects, because it does not protect the respondent’s 
privacy in case of a “yes”-answer at all. All the other 
procedures protect a “no”-answer more than necessary. 
Therefore they may be used, but they cannot achieve the 
efficiency of ST4. 

If the membership of both U, and U% is sensitive, so 
that the variable is sensitive as a whole (for instance: U, = 
set of married people, who had at least one sexual 
intercourse with their partners last week; US = U —-U,), 
Ai opt S Ao,op < © applies. In this case neither the direct 
questioning on the subject nor design ST4 can be used 
because they are not able to protect both possible answers. 

The other designs are applicable for such topics, but 
Wamer’s design cannot achieve the efficiency of the others, 
If Ay oot < Ao, opt: The reason is that this design always 
protects the respondent’s privacy with respect to a “yes’’- 
answer equally to a “no”-answer. But if A, 4. = % 
despite to the claims of some publications in the past (see 
for instance: Greenberg ef al. 1969, page 526f, Mangat and 
Singh 1990, page 440, Singh ef al. 2003, page 518f) there is 
not one randomized response technique that can perform 
better than Warner’s technique S72 with the optimum 
design parameters p, and p, according to Table 2. For 
ST7 this is only valid for >, or <Ao,on- Therefore ST7 
is the perfect supplement of S72, for which the very 
opposite is true. 


0, opt 


Statistics Canada, Catalogue No. 12-001-X 


148 


Table 2 


Quatember: A standardization of randomized response strategies 


Optimum design parameters for given 4, and A, and different types of sensitivity of the variable under study 


Questioning design (Subject category) 
of aes (Ea) 
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All others of the designs of Table 1 like S711 or ST14 
can perform equally efficient for A, 1 < Ao, on < ©, if the 
design parameters are chosen according to the restrictions 
(14) to (15). Among them Greenberg ef al.’s strategy with 
known 1, (S73) has on the one hand the advantage over 
Warner’s design to be able to perform optimally also if 
Ar opt < Ao,opt: On the other hand, however, it has the 
disadvantage (like S76), that the size 1, of subpopulation 
U, is completely predetermined (or at least bounded by an 
interval), if we want to achieve the optimum efficiency. This 
means in practice, that we have to find a subpopulation not 
related to the possession and nonpossession of attribute A 
and of appropriate relative size to be able to achieve the 
estimator’s optimum accuracy. In principle this also applies 
to S79, ST10, ST12 and S713, but looking at the 
presettings of design parameter 7,, it turns out that ST9 
and ST10 as well as S712 and S713 _ perfectly 
complement each other so that in fact any subset U, of the 
population can be used. Finally the most complex special 
cases, ST15 and ST16, of our standardized randomized 
response strategy can both be used with any subpopulation 
U, <U toachieve the best performance. 


5. A real-data example 


An empirical study was carried out to demonstrate the 
applicability of the strategy as a questioning design. For this 
purpose the population of 80 students, who attended the 
author’s course on “Statistics II’ at the Johannes Kepler 
University in Linz (Austria) during the spring term of 2009, 
volunteered for the survey. The subject under study was 
academic cheating behaviour. To this end cheating was 
defined as any behaviour, that was not allowed in the 
written exams (including just looking at the test scripts of 
other students or the use of forbidden documents). It is 
beyond doubt that this subject is sensitive for such a 
population. Moreover during the survey all of the students 
were sitting in one lecture room. The parameter of interest 
was the proportion of the population of students, that fudged 
on at least one of the exams of the previous semester 
(including the exam of the author’s course on “Statistics I’). 
Therefore it is beyond reasonable doubt to assume, that 
direct questioning on the subject would have resulted into a 
substantial underestimation of this proportion. An empirical 
study of Scheers and Dayton (1987) for instance showed 
very small proportions for almost all different cheating 
behaviours asked, when the subject in question was asked 
directly. The use of Greenberg’s randomized response 
strategy ST3 lead to a significant increase of these 
proportions (ibd., page 68). 

Apparently, for the variable of interest the membership 
of group U,, formed by the “cheaters”, is sensitive, but not 
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the membership of the complementary set U% . Therefore in 
accordance with the recommendations of Section 4 we 
decided to use questioning design S74 for our survey and 
to compare it with Warner’s strategy S72. The A -values 
of loss of privacy were fixed at A, = 4 and A, = ©. From 
Table 2 we calculated p, =0.75 and p, =0.25 as the 
variance-optimum design parameters of ST4. To achieve 
these probabilities the students were asked to throw two dice 
without showing the result to somebody else and answer in 
a questionnaire the question “Did you cheat at the exams at 
least one time?” only if the sum of the numbers on the dice 
was 5 to 10. Otherwise they should just respond “yes”. 

Previous to the survey some effort was made to explain 
the consequences of this randomization strategy on the 
privacy protection. After giving the answer on the first sheet 
of the questionnaire, only these sheets were collected. 63 out 
of the 80 persons answered “yes”. 20 of 80 students were 
expected to do so, because they received the “say yes- 
instruction”. Therefore expected 43 of 60 other students 
should have answered “yes” on the sensitive question. The 
estimator for 1, is given by 


a ST 


asrs _ fy ~ Pa _ 0.7875 ~ 0.25 
; DP, 0.75 


= 0.716. 


For this population survey the estimated variance of ft, 
is then 

gee 
ee el Lele oee 

n- Py 


VR) 


After this questioning design was completed, the students 
were asked directly on the second sheet of the questionnaire, 
whether they had truthfully answered the first question or 
not. Only four students said that this was not the case. This 
means, that — if that’s true — it is likely that 4 more students 
did actually cheat. The next question to answer was, if they 
would still cooperate, if p, (of ST4) would be higher than 
0.75. 32 of 80 students agreed to do so, but the others did 
not. Obviously (at least) four of them did not cooperate 
when p, was 0.75. 

Finally, Warner’s technique was applied with the same 
sensitive question as ST4 before. To come close to a A, - 
level of 4 — indicating the same loss of privacy as to a “yes”- 
answer for both questioning designs —, the sum of the 
numbers of two dice had to be 3 to 9 to apply a design 
parameter p, = 0.805. The A -measures of loss of privacy 
for this choice are given by A, = A, =4.143, indicating a 
slightly higher loss of privacy compared to ST4. With a 
probability of 0.805 the students had to answer “Are you a 
member of U,?” and with the remaining probability the 
alternative “Are you a member of U4? ”. 

Now only 38 of 80 persons gave a “‘yes”-answer. This 
results in an estimated proportion of “cheaters” of 
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- ST 


ged 8, Mae pprateOATS = ON 94 
TU 4 = —_—_——_—__ - -ererororv—xX— 


= 0.4590. 
P, — Pr 0.61 


Additionally to the slight increase of the objective loss of 
privacy there is another reasonable explanation for this 
significantly lower result. Although A, did not change that 
much, some test persons must have been irritated by the 
raise of p, up to 0.805 after being asked for ST4, if they 
would still cooperate, if p, would be higher than 0.75. Not 
being able to distinguish between the loss of privacy caused 
by different design parameters in different questioning 
designs, some of the “cheaters” did not want to answer 
truthfully again. Just to demonstrate the effect of the 
different questioning designs on the efficiency of the 
estimation process we calculate the estimator of the variance 
Dt gi, 


dial Sy ailiapersoy bragyi ee 
n- (2p, — 1) 


The reason for this considerable increase of the estimated 
variance is, that Warner’s strategy does protect a “no”- 
answer always in the same way as a “yes”. Since in our case 
a “no”-answer does not have to be protected at all, this 
unnecessary protection has to be paid in terms of accuracy. 


77 /AST2 
Vt pn) = 


6. Summary 


Randomized response strategies have originally been 
developed to reduce the nonreponse as well as the untruthful 
answering rate for sensitive subjects in sample surveys, but 
they can be applied as masking techniques for public use 
microdata files as well. The standardization of these 
techniques for the estimation of proportions developed in 
this paper provides an opportunity to derive a general 
formula for the variance of the estimator under probability 
sampling. Different questioning designs, partly published, 
partly — to the best of our knowledge — unpublished up to 
now, can be regarded as special cases of the standardized 
strategy (see Table 1). For the purpose of a comparison of 
the accuracy of these designs it is essential to include the 
levels of privacy protection offered by them in our 
considerations. Doing this by means of the “A -measures” 
of loss of privacy explicated in Section 3 a completely new 
picture has to be painted in comparison to almost all 
publications in the past as far as the author knows them. It 
turns out that the identifying or sensitive subjects have to be 
classified into different categories in order to find the 
variance-minimum questioning designs for a given privacy 
protection (see Table 2). The first category consists of 
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subjects, which are not sensitive at all. The second 
comprises topics, where only the possession but not the 
nonpossession of a certain attribute is embarrassing to the 
respondents. The last category is formed by subjects, which 
are sensitive as a whole. 

For subjects out of the first category it is clear enough 
that no strategy can be more efficient than the direct 
questioning on the subject (ST1 of Table 1). 

Concerning topics of the second category there is just 
one design available, that can achieve the minimum 
variance of the estimator. This is the questioning design in 
which each respondent either with probability p, has to 
answer the question on membership of the sensitive group 
or with probability 1— p, is instructed to answer “yes” 
(ST4). All the other special cases of the standardized 
strategy protect the interviewee’s privacy not only in case of 
a “yes”-answer like ST4 does, but also in case of a “no”- 
answer. Therefore their performances cannot reach the 
minimum achievable level. 

For subjects out of the third category it is shown, that 
contrary to the claim of other publications, there is not one 
single strategy available that can perform better than 
Warner’s of 1965 as long as the membership of the 
subgroup under investigation is equally sensitive to the 
membership of its complement. A lot of other designs are 
equally efficient as Warmer’s but not a single one is more 
efficient. 

For the variables of this category, where the membership 
of one group is sensitive, but not equally sensitive as the 
membership of the complementary one, the situation 
changes dramatically: Compared under the same levels of 
privacy protection Warner’s technique is not able to achieve 
the best achievable performance of the standardized 
randomized design anymore, whereas many other strategies 
can. For some of the designs including the question on 
membership of a nonsensitive subpopulation not related to 
the attribute under study, it is required to find an adequate 
subpopulation of predetermined relative size. Other designs 
can be used with subpopulations of any size and are 
therefore more practicable. Therefore a data collector or 
publisher could select that one of the equally efficient 
designs, that seems to be more easily applicable than the 
others. 
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Treatments for link nonresponse in indirect sampling 


Xiaojian Xu and Pierre Lavallée ' 


Abstract 


We examine overcoming the overestimation in using generalized weight share method (GWSM) caused by link 
nonresponse in indirect sampling. A few adjustment methods incorporating link nonresponse in using GWSM have been 
constructed for situations both with and without the availability of auxiliary variables. A simulation study on a longitudinal 
survey is presented using some of the adjustment methods we recommend. The simulation results show that these adjusted 
GWSMs perform well in reducing both estimation bias and variance. The advancement in bias reduction is significant. 


Key Words: Weight share method; Nonresponse; Indirect sampling; Longitudinal survey. 


1. Introduction 


Indirect sampling refers to selecting samples from the 
population which is not, but it is related to, the target 
population of interest. Such a sampling scheme is often 
carried out when we do not have sampling frames for the 
target population, but have sampling frames for another 
population which is related to it. We call the latter sampling 
population. For an example in Lavallée (2007), we consider 
the situation where the estimate is concerned with young 
children belonging to families, but we only have a list of 
parents’ names as our sampling frame. Consequently, we 
must first select a sample of parents before we can select the 
sample of children. In this typical indirect sampling 
situation. The sampling population is that of parents while 
the target population is that of children. We note that the 
children of a particular family can be selected through either 
the father or the mother. Figure 1 provides a simple 
illustration for this indirect sampling scheme (Figure 1.2, 
Lavallée 2007). 

There is a sizeable amount of literature concerning 
estimation problems that are associated with indirect 
sampling, a few of which we name here. Initially, estimation 
methods for production of cross-sectional estimates using 
longitudinal household survey are discussed in Ernst (1989). 
This study presents weight share method in the context of 
longitudinal survey and also shows that this method 
provides an unbiased estimator for the total for any 
characteristic in the population of interest. Kalton and Brick 
(1995) conclude that such a method also provides minimal 
variance of estimated population total for some simple 
sampling schemes for the longitudinal household panel 
survey. Lavallée (1995) extends weight share method in a 
completely general context of indirect sampling which 
includes longitudinal survey as its particular example, called 
generalized weight share method (GWSM). This work 
justifies that this weighting scheme provides unbiased 


estimates irrespective of sampling schemes in obtaining a 
sample in the sampling population. As with any other 
weighting scheme, in the process of GWSM implemen- 
tation an adjustment for a variety of nonresponse problems 
must be made. Lavallée (2001) provides adjusted GWSM 
incorporating possible total nonresponse problems in 
indirect sampling. In indirect sampling there is another type 
of nonresponse called link nonresponse, termed by Lavallée 
(2001) as “relationship nonresponse,” which is associated 
with a situation where it is impossible to determine, or 
where one has failed to determine, whether or not a unit in 
the sampling population is related to a unit in the target 
population. Lavallée (2001) points out the problem of 
overestimation in using GWSM when link nonresponse 
occurs and leaves finding suitable adjustment of GWSM for 
link nonresponse as a rather open question. This present 
study focuses on developing treatments of estimation bias 
caused by such link nonresponse. 
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Figure 1 Indirect sampling of children 
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The rest of this work has been arranged in the following 
sections. Notation and the problem defined are described in 
Section 2. We propose a few modification methods in using 
GWSM incorporating link nonresponse in Section 3. A 
simulation study using a real life data set is presented in 
Section 4 with a few closing remarks in Section 5. We note 
that we show the advances of the new methods provided in 
this paper through a simulation study while other theoretical 
contributions relevant to this problem can be found in 
Lavallée (2002), Deville and Lavallée (2006), and Lavallée 
(2007). 


2. Notation and problem 


We use U% and U® to denote sampling population and 
target population respectively. Then, U~% is the population 
related to U*® with a known sampling frame. We let 
s4,M%, and m* be a selected sample from U%, the 
number of units in U%, and the number of units in s“ 


respectively. We Se nC to represent the selection 
probability of vi ities in U* with 1 >0 and 
ss 1g =m*. We also make use of the notation: 


j=l 
M?, N, Ue Ma M? to be the number of units in U’, the 
number of clusters in U*, the i” cluster of U? with 
U",U? =U", and the ‘fo SERRA HE RS one 

We define /, , as an indicator variable of link sxusisncs: 
Lin =] indicates that there is a link between j" unit in 
emeney ete ae U?, while J,,=0 indicates 
otherwise. We also define me ; as the total number of links 
See between unit / of U* and units of U’, ie., 

1 Oe =e, 4. Let L? be the total number of lls 
2 between units of U“ and units of U’, ie, 
hb, iS Helis: We mec the value of the sues 
for the k™ unit of i" cluster in population U? by y,, and 
the bole! of all y,s by Y*. Then, 
y* = oa =1 Vik: 

We let Q? denote the clusters in U? where there is at 
least one unit ik such that /,, =1 forsome j™ unit ud ae 
and we say that it can be identified by units j in s“, ie., 
such i satisfies L; = i oe 1, ,> 0. The erie Fi 
clusters in Q? is n. After RS per we relabeled the clusters 
in Q? as i=1,2,...,n. We let w, refer to the estimation 
weight assigned to k" unit of i" cluster, Q4 refer to the 
set of units in U“ that have links to some units in U? with 
i€Q?, and 7 refer to the set of units in U7 that have 
links to some units in Q”, ie, Q4 = {j|Dieq L?, #0}. 
We use s** to indicate the set of units in s* that have links 
to some units in U? with ic Q?. We let 77,77, and m4 
denote the number of units in Q%, the number of units in 
Q4, and the number of units in Ss. respectively. Finally, we 
make use of the following three PetCo let ¢, be the 
indicator variable of being selected in s4 t, =1 haicaes 


we have 
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that ap unit in U“ isin s* and t; =0 indicates suena 
let a be the inaicator variable SE Dole included in s“ oe 
units in Q4: 1; =1 indicates that j" unit in Q* is in s* 
and ‘ts =0 Fito otherwise; ae let ti ; be the ce 
Sealer being included in s/ for vane iin OO aol 
indicates that j" unit in Q4 isin s/ and ft; ,=1 faeries 
otherwise. 

Our goal is to estimate the total Y’, the parameter of our 
interest, for target population U? which is divided into N 
clusters. In order to do so, we select a sample s* from U“ 
with selection probability m . Then we identify Q? using 
1, 4, #0. All units of the Sisto in Co Mate surveyed where 
y,, and the set of /, ,, are measured. 

By applying the GWSM, an estimation weight w, will 
be assigned to each unit k of surveyed cluster i’s. Such 
weights can be chosen in an appropriate manner so that the 
estimator of Y”: 


M 
y’” my Des Wi Vik (1) 


performs well in estimating Y”. 

We are interested in estimating the quantity Y? using 
Y*. According to Horvitz and Thompson (1952), let w, be 
inverse of selection probability, ,, of the k individual 
of U? in the target population. Then Y® gives an unbiased 
estimator for Y’. However, the computation for 7, is 
difficult or even impossible in the present case, due to the 
complication in the indirect sampling scheme. Therefore, 
GWSM is introduced to address this issue. For readers’ 
convenience, here we outline the GWSM in computing the 
weights for each cluster that has been observed. 


Step 1: Provide the initial weights wi, 


M4 ft 


Wi = oe Dik ahs (2) 
j 


jel ua 
Step 2: Compute L? 


es Di: (3) 
are Za" j,ik? 


w, =+L_; (4) 


Step 4: Set w, = w, forall kin i cluster. 
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It follows Theorem in Section 3 of Lavallée (2001) that 


M4 t 
arte 
Jit A yy8 
j=l Ti i 
Vik (5) 


offers an unbiased estimator for Y" provided all links / jik 
can be correctly identified. The estimation weights assigned 


in (5) are 


Wiz = 
M4 
if. 
Baas 
5, Li pers 
j=l 1 ; : : ms B. (6) 
EB , forall units & in cluster i when i in Q”; 
0, when i is not in.Q?. 


A simple example is illustrated in Figure 2. We aim to 
estimate the total Y" linked to the target population U’. 
Suppose that we select the units j=1, and 2 from U“. By 
selecting the unit 7 =1, we survey the units of cluster i =1. 
Likewise, by selecting the unit 7 =2, we survey the units 
of clusters i=1, and 2. We therefore have Q? = {1, 2}. For 
each unit k of clusters i of Q?, we calculate the initial 
weights w, in (2), the total number of links existing 
between units of U“ and units of U)’, L?, and the final 
weights w,. Then, according to (5) the resulting estimator 
for Y” is as below (see Lavallée 2007, pages 17-18 for 
more details): 


fre bsstlerwitol: Ala, yet Le hb bby Mol (7) 
nA ru bao afar ee eee 


1 Pe 

We note that for the estimator with known /, ,,, the only 
assumption for unbiasedness is to have L? >0 for all 
clusters i’s in U®. That is, every cluster of the target 
population must have at least one link from U“*. We know 
that if some links were missing, then the estimator (5) would 
be biased. When link nonresponse occurs, as indicated in 
Lavallée (2001), i can not be determined. Traditionally, 
using total links observed to replace this unknown quantity 
results in overestimation on Y° since some link com- 
ponents are actually missing in summation L?. Our pro- 
posed study focus is on just such a problem, and we attempt 
to adjust the estimation weights w, by estimating L? so as 
to obtain a better performance of estimation on Y°. 
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Figure 2 Example of links in indirect sampling 


3. Treatments of biased estimation problems 


As indicated in Section 1, the biased estimation using 
GWSM occurs due to link nonresponse problems. In this 
situation, not all of the composition in L? can be identified 
or observed. Although the links between units in s* and 
units in U* can normally be determined in practice, the 
parts of links outside s* are often difficult or even 
impossible to identify. We say that such units have missing 
links with U®. Let A* =Q4\s% be the set of units with 
possible missing links. Then, 


M? M? 
L = m Dake, re » aie (8) 
es k=l JIN Les 


If we carry out the GWSM without taking these missing 
links into account, we use the total of observed /, , as Ca 
instead to compute Y” using 


M? M? 
i = oy Daring Fa » Da tea; (9) 


jest k=l jead k=] 


where A“ is a subset of A“ and only contains the units 
whose links are observed. The cost is overestimation of Y” 
in using (5) since 


JE Peat bone 
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We suggest a few methods for applying GWSM under 
consideration of link nonresponse by estimating L,; . 


3.1 Estimating jis without availability of auxiliary 
variables 


3.1.1 Estimating L? by proportional adjustment for 
each individual cluster (Method 1) 


To address the link nonresponse problem, we focus on 
estimating L? using the known information about the links 
within s*. To compute the weights in (6) using GWSM, we 
only need to estimate L? for those i¢ Q?. For any ie’, 


Te 
Ls ite (10) 
j=l 


A general estimator for this total can be expressed as 
ifs 
rB B 
LP nls as (11) 
| 
where Wii is a random woe that takes the value Wi =i) 
if j is not in the sample s“. For oe ie Q?, we use Laie 
known link information Benieen s and U/ to estimate 
the link information between Q4 and U,’. The expectation 
of L? is 


Re 
E(L?) =) E(w; Lj. (12) 


By comparing (10) and (12), it can be observed that i is 
unbiased for L? for any weighting scheme with E (we )=l 
for all /. 

First of all, we adopt the Horvitz-Thompson estimator 
(Horvitz & Thompson 1952), also called a estimator 
(Samdal, Swensson, and Wretman 1991). Note that, by the 
definition of Q7,Q4 554 for all i. We imitate a procedure 
for estimating the number of links in Q using that in s/*. 
The procedure is to Saek) a “sample” s“ from the 

popnlanens Q4. Let Tw pare the probability of 7 (which is 
in Q* ) being included i 7 ‘sf . Then, let 


iL 
La ees 
Wii = 
0, 


According to Corollary 3.1 in Cassel, Saémdal, and Wretman 
(1977), this weighting scheme provides an unbiased 
estimator for L’. We have 


Ait . A 
Wis Ws, 

13 
dais MQ s. ee 


dba Bit ; 
x Bay ee (14) 
ja 


It provides us with an asymptotically unbiased (proof 
follows) estimator of Y?: 


Statistics Canada, Catalogue No. 12-001-X 


aE See (15) 
ji 


In order to show its unbiasedness, we employ Taylor’s 
expansion. According to Corollary 5.1.5 (Fuller 1996), we 
obtain 


] \ 


Le ae (Lh 1) Oa) 
ae A es Ok ie! 
“Ty ; ~L,)+0,(7)). 

It follows that 


: ee : 
plim eae, +2) 0 


Therefore, by Theorem 5.2.1 (Fuller 1996), the limiting 
distribution of n'/7[1/ ih ] is the limiting distribution of 
jt NWI bad OA - id )]. We note that Y? is a function of 
both random variable: ¢,, and random variable: Pee 
therefore we denote the expectation og Y? with respect to 
t, by E, (-) and that with respect to t; ; by Ey _0). Hence, 


awitaeale we have 


E(Y*) 


n 1 5 t M? 
=D E, 58 Ba yay (16) 
n ) . t, MP 
=E,| 2) ye Ehime [2a Ye (17) 


According to Lavallée (1995), E, meg ®)-Y*. Therefore, 
Y* is an approximately unbiased estimator of Y?. 
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Now we need to compute Tw, ;- It is a function of Le yet 
it depends on how s/* affects on U/’, therefore on 4. 
Such an effect is difficult to track and varies from case to 
case; however, we can give a general estimate of it. The first 
approach we propose in this paper is to estimate selection 
probability, ;, using the proportion of the units in s% 


which take in Q*. Namely 


A 
Li) _ ™,; 
Tj, i ae re (18) 
i 
Therefore, 
ike je te 
rB(1) _ ji i 
L; a, aL) 
j=l Ji 
A 
Ts m; : 
ar aDs Lisi (19) 
Lge es 
and 
M4 
> 1 i un 
n al 7A M? n M? 
7B) _ = J = (1) 
Ne => TA mi De Da ae Vike (20) 
ist LW RB Oke i=] k=l 
oe = Li 
i j= 
with 
4 7B 
A 
(1) m;" j=l Tj 
ig er (21) 
T; 


We revisit the example in Figure 2, assuming that there 
are two link nonresponses that happened between the unit 
j =3 in U% and the units k =1,2 of cluster i=2 in U’. 
If we use the GWSM without adjustment in (5), the 
resulting estimator for Y? is no longer (7). We have instead 


ah eer 7 ib ya y 
Y DR ae Vit 2\ nt x 12 


1 1 1 
+— +— +—y,,, a2 
, ya , y22 , Vo; (22) 


which is biased. In order to apply (20), we first compute 
m“/T,4. Then the resulting weights using Method (1) in 
(21) for this example is shown in Table 1. Therefore, this 
modified method provides the estimator: 


si 1 a 1 per \\ i ] . 
Y 5 x mt an > 1 oy, 12 
Selon supuilag§ papicinals (23) 
an 2) re, 22 2nd 23> 


which is less biased than (22). 
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Table 1 
Initial weights, total number of responded links, and final weights 
from (21) 


Link Suite ated nak neste Tia wilt 
Lintite tid 1 2awl2 1 1/20/nf+1/n4) 
I Dans 1 2 19/20 / nt £1) x2) 
PP MYVEOS WoGnissing) G1 S Roles 1/9 1/2n3 
Jar? line one link» lo. | 12 1/2n4 

is missing) 
Peli alah 0 gee omer tal? 1/2n4 


3.1.2 Estimating iby by overall proportional 
adjustment (Method 2 ) 


In the previous approach, the information regarding m/ 
and 7.“ is needed for every i. Suppose we ignore the 
variation of Q7 among all i, then we simply propose that 


bomahaate 
| ees (24) 
jar My 


using link information in s* to estimate the link 
information in T*, where bi being the indicator variable 
for being in s* from Q*. Now we need to compute qs 
Again it is a function of 1 and yet it depends on the 
complexity of effects of s* on Q?, hence to Q*. While 
the computation is difficult and varies from case to case 
without a general form, we can usually give a rough 
estimate of it. 

The second approach we propose in this paper is to 
estimate 1; using the proportion of the units in s* which 
appear in 0%, ie., Ti =m*/T%. It informs us that 


2?) = eS, LE (25) 
i = firs ji 
mM j= 


For simple random designs with or without stratification, 
ibe provides an unbiased estimator for L?. For more 
complex designs, it provides a model-based unbiased 
estimator under assumption (A) as follows: 

(A) Suppose that for any cluster i, the average of total 
existing links associated with all units in the sample s“ is 
the same as that of existing links associated with all units in 
Oo en 


i B ise B 

Da Lii dD Lii 

Ja = Be, : (26) 
m 


So, the estimation weights are provided by 
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M‘ t 
BS 
pet pe 1s a 
= 
Whe) yl?) : /, for all units & in cluster i. 
i i TA M4 


(27) 


It follows that Y’ can be estimated by 


Aion 
poratys LS 94 = ap Ves 08) 
=| 


TA ¢ em 
me Ae 


j=l 


We recall the example in Figure 2 with two link 
nonresponses that happened between the unit j =3 in U“ 
and the units k =1,2 of cluster i=2 in U®. In order to 
apply (28), we first compute m*/T*. For this example, we 
have m4 =2, and T* =3. Then the resulting estimator for 
Y” using the adjustment Method (2) for this example is 


Fe Pans 6 We eas ov) aes 6 
s AW ee Me ne “a a4 nee ny *12 


play “by eal (29) 
2 2 2 

Therefore, this adjustment made in (28) is different from 
Method (1) for this example. 

We know that var(Y2"°”) = var{E(¥20%” | 54)}4 
E{var(Y?"°” | s4)}. The inner expectation and variance 
(conditional on s*) are taken over all possible sets of 
“responding” /,,, given the sample s* while the outer 
expectation and variance are taken over all possible sample 
s*. Generally, the adjustments made above will not 
eliminate the second term which depends on_ the 
randomness of /, 


3.2 Estimating L? with availability of auxiliary 
variables 


3.2.1 Estimating /; , using logistic model 


The estimation methods for L’ proposed in Section 3.1 
are simple to apply and do not need additional information. 
However, sometimes the assumption can be violated which 
results in an undesirable estimate. For instance, jis ; May 
depend on some characteristics of unit / and cluster i. 

We assume that the probability of a link between a unit 
in sampling population and a unit in target population 
depends on some auxiliary variables through a logistic 
regression model. We may estimate this probability function 
so that the estimation of the quantity of interest in the target 
population is desirable. Let P., = P(/,, =1) which is 


Jj, ik 
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affected by some variable vector x4 in U* and x}, 
in U?. 
We may fit the logistic model 


Ps 
log} —*_ |= a’x* +b'x3 
Drse Pi ig 


using the observed links and their corresponding 
characteristic variables. The unknown parameter vectors a 
and b can be estimated. Then, for those /),s which can 
not be identified we suggest to impute them with their 
probability estimates: 


(30) 


a'x4 +b'x} 
e 
5 mer en (31) 
lie 2% 
where (a,b) is an estimator for (a,b), for instance, we use 
the weighted maximum likelihood (pseudolikelihood) 
estimator. We then have 


fey iene 
Lo = Li,+ Li 
EIN jeQ4\(s4 VAG) 
iV Fa a'x4+b'xi 
De a Wee —- (32) 
? a'x4 + x5 
jest jeO4\(s4ad) k=l Llt+e 


After replacing L? with jing ) in (5), (5) provides us with a 
consistent estimator for Y® when the model specified in 
(30) is correct and (a,b) is consistent. Note that there are 
alternatives for the logistic model, such as logit and 
complementary log-log models. See Draper and Smith 
(1998) for details. Their research also states that the choice 
of which model should be employed is not always clear in 
practice. 


3.2.2 Directly estimating I bre use log-linear model 


We consider that there is a variable vector x? which 
affects the value of L?. This indicates that the total number 
of links in a cluster only varies according to the 
characteristics of the cluster itself. Using the log-linear 
model, we can propose (33) below: 


lok )=0' x= (33) 
If the fit is reasonable, L? can be estimated directly by 
ieee (34) 


where 6 is an estimator for 6. When 6 is consistent then 
after replacing L? with Bi in (5), (5) provides a 
consistent estimator for Y’. We note that 1? might be 
non-integer valued, and therefore might have to be rounded 
to the nearest integer value. 
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When the production of cross-sectional estimates at a 
particular point in time after the initial point is also of 
interest in a longitudinal survey design, it becomes a 
practical example of an indirect sampling problem. Since 
the population changes over time, the target population is 
not the same as the initial population which the longitudinal 
sample is selected from. In this section we will use Survey 
of Labour and Income Dynamics (SLID) as an example to 
demonstrate the performance of one of the estimators we 
introduced in Section 3.1. 

The sample design for SLID is detailed in Lavallée 
(1993). Some terminologies we use in this report - such as 
cohabitants, initially-present individuals, and initially-absent 
individuals - follow Lavallée (1995). Initially-absent indi- 
viduals in the population are individuals who were not part 
of the population in the year the longitudinal sample was 
selected, but are considered in the later sample; included 
among these are newborns and immigrants. After the initial 
year of selection, the population contains longitudinal 
individuals, initially-present individuals and initially-absent 
individuals. Focusing on the households containing at least 
one longitudinal individual (i.e., longitudinal households), 
initially-present and initially-absent individuals who join 
these households are referred to as cohabitants. 

In this specific example, U* is the population at the 
initial year, say yr, of the longitudinal survey, and U” is 
the population at any of the following years, say year yr,, 
after the initial year. The sample s“ is all the longitudinal 
individuals. L,; is a binary variable; it values 1 if 
individual j lives in i" household at yr,;0 otherwise. L? 
is the total number of longitudinal persons and initially- 
present cohabitants at yr, who lives in i household at 
BAe 

For a longitudinal individual the link would be one to 
one. For cohabitants there is a significant possibility that this 
link will be impossible to identify a few years past the initial 
year, for reasons such as new birth and immigration; further, 
the greater proportion of cohabitants occupying the target 
population, the larger this possibility becomes. For instance, 
in survey panel 3 in SLID, cohabitants represent 7.8 
percents out of 47,377 individuals in the year of 2000 which 
is one year after the initial year. This increases to 13.87 
percent in the year 2002 (3 years later), and 15.22 percent in 
2003 (4 years later). We can see that the link nonresponses 
can not be overlooked in such a significant proportion of 
cohabitants. Due to the availability of observed information, 
we implement the approach of estimating L? by two kinds 
of proportional adjustments, which we proposed in Section 
3.1.1 and 3.1.2. In order to test the performance of the 
estimates obtained by these approaches, we carry out a 
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simulation study using SLID data. Cross-sectional esti- 
mations for four income variables are of interest for the year 
of 2003. These four variables are: total income before taxes; 
total income after taxes; earnings (includes wages and 
salaries before deductions and self-employment income); 
and wages and salaries before deductions (also called 
employment income). We are interested in the total of the 
population incomes for all these variables. These four 
quantities of interest have been estimated at both the 
national level and the provincial level. 

For a longitudinal survey, the total number of links in 
cluster i are generally not more than the total number of 
individuals in this cluster and not less than the number of 
longitudinal individuals in this cluster. Since 7,’ is 
unknown, we replace 7,? by M? in (5) in our simulation 
study. 

First, we assume that the links between all units selected 
in the initial year (1999) and all units in the whole 
population in 2003 are correctly specified. Then we 
compute the totals using GWSM. We use it as our 
estimation target, the “truth.” 

Second, we randomly take away 50 percent of the links 
associated with initially-present individuals by setting up at 
random some initially present cohabitants as initially absent 
ones. The number of links taken makes up approximately 
6.3 percent of the total population with which we are 
interested, with a size of 30,224. Without any adjustment, 
we recalculate the estimates using GWSM. We use it as our 
estimation benchmark, the “placebo.” 

Third, we estimate the same quantities using GWSM 
with proportional adjustment approaches, Method (1) and 
(2) in Section 3.1, to see whether the estimates are close 
enough to the “truth” and how much improvement these 
adjustments make. 

This simulation study using SLID data demonstrates that 
the proposed method performs very well in overcoming the 
overestimation problems that arise from link nonresponse. 

We denote 


(35) 


Then, using Method (1) and (2) in Section 3.1 we 
estimate Y? by 


n m2 M? 
C = -- woes ys Vik» (36) 
i=] fé k=] 


and 


Statistics Canada, Catalogue No. 12-001-X 


160 Xu and Lavallée: Treatments for link nonresponse in indirect sampling 


m4 n MP 
y* 2 
re aM oh ss Vik» (37) 
i=] k=] 

respectively. 

We note that w*™" is the average weight of longitudinal 
persons who live in i household at yr,. Therefore, it is 
also reasonable to use median weight: 


an 


1 
wen — the median of — ony ee es atu’ (38) 

1, 

i 


instead to enhance the robustness of the estimates. Namely, 
we estimate Y* as well by 


n m4 M2 
7B) ee j median 
yeina z » ez W; ye Vik» (39) 
k=l 


i=] i 


and 


a 
B(2) 


if median ~ 7,4. Wee x Vik: (40) 
1 ber 
The comparison for these proposed methods with and 
without incorporation in nonresponse problems both using 
mean and median weight within each household are 
presented in Tables 2-5. 
The next four tables give the result for the performance 
of our estimate using relative error defined as: 


Table 2 

Total income before taxes (in Canadian dollars) 

Province Estimates by GWSM Estimates by GWSM 
without missing links with missing links 

NFL 9,261,958,108 9,788,749,735 

PEI 2,720,448,008 2,858,506,466 

NS 18,277,017,251 19,573,546,299 

NB 15,297,159,323 16,281,178,934 

QC 1.37839E+11 1.69664E+1 1 

ON 2.895E+11 3.07642E+1 1 

MA 23,436,397,548 25,043,168,032 

SK 20,185,285,649 21,595,804,296 

AB 69,063,402,292 74,576,35 1,600 

BC 81,749,374,346 86,593,614,506 

National 6.8733E+11 7.33617E+11 

Table 3 

Total income after taxes (in Canadian dollars) 

Province Estimates by GWSM Estimates by GWSM 
without missing links with missing links 

NFL 7,846,587,557 8,287,351,908 

PEI 2,300,092,795 2,416,503,441 

NS 15,154,508,564 16,257,679,161 

NB 12,878,350,198 13,718,260,686 

QC 1.27632E+11 1.37514E+11 

ON 2.3788E+1 | 2O3073EFT1 

MA 19,541,510,220 20,877,377,918 

SK 16,894,929,025 18,073,635,883 

AB 57,466,974,767 62,055,315,246 

BC 68,710,569,670 72,770,595,462 

National 5.66306E+1 1 6.05044E+1 1 
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estimate - “truth x 100%. 
ruc 

Estimates by adjusted Estimates by adjusted 
GWSM using mean GWSM using median 
9,317,420,236 9,304,530,248 
2,735,943 ,043 2,734,922,451 
18,140,076,618 18,067,144,557 
15,291,696,585 15,236,482,035 
L.56533E7U1 1.56405E+11 
2.85409E+1 1 2,8. 99E+1 1 
23,632,717,226 23,553,543,216 
20, 163,683,598 20,095,359,071 


68,716,661,193 
81,387,640,982 


68,582,541,733 
81,248,680,715 


6.8286E+1 1 6.82356E+11 
Estimates by adjusted Estimates by adjusted 
GWSM using mean GWSM using median 
7,892,754,014 7,882,437,105 
2,314,256,124 2,313,544,320 
15,080,155,194 15,020,088,623 
12,894,700,593 12,849,252,205 

1.271 18E+11 L26999E +14 
235192ER) 2.3534E+11 
19,713,628,649 19,649,142,217 
16,890,410,993 16,834,787,407 


57,183,814,491 
68,43 1,531,373 
5.63958E+1 1 


57,073,904,623 
68,309,055,749 
5.63518E+11 
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Table 4 

Earnings (in Canadian dollars) 

Province Estimates by GWSM Estimates by GWSM Estimates by adjusted Estimates by adjusted 
without missing links with missing links GWSM using mean GWSM using median 

NFL 6,433,112,169 6,837,522,157 6,541,306,193 6,530,174,122 

PEI 1,898,192,704 2,019,341,995 1,964,066,449 1,962,669,664 

NS 12,772,667,160 13,809, 197,160 12,999,111,234 12:939.785,579 

NB 11,250,688,811 12,030,378,710 11,411,530,716 103 90,222,533 

QC 1.18878E+1 1 1.28949E+11 LA979TE+11 betS717E+11 

ON Zo2ISTIETLI 2.43404E+1 1 2.26812E+11 2.27092E+1 1 

MA 17,560,695 ,670 18,995,682,322 18,066,353,153 18,001,882,362 

SK 15,159,319,031 16,340,668,148 15,381,733,004 15,319,210,228 

AB 56,152,023,359 61,059,244 ,608 56,540,145,524 56,418,889, 147 

BC 60,532,655,979 64,499,398,960 61,192,920,832 61,085,986,951 

National 5.28214E+11 5.67945E+11 5.3199E+1 1 S21 722E+11 

Table 5 

Wages and salaries before deductions (in Canadian dollars) 

Province Estimates by GWSM Estimates by GWSM Estimates by adjusted Estimates by adjusted 
without missing links with missing links GWSM using mean GWSM using median 

NFL 6,180,713,343 6,572,345,010 6,283,079,555 6,272,429,515 

PEI 1 636,344,440 1,747,755,878 1,713,809,312 1,713,157,676 

NS $2,327220157; 13,341,912,666 12,579,S19,733 12,521,159,025 

NB 10,742,381,379 11,508,445,078 10,961,105,589 10,921,102,477 

QC 1.08636E+1 1 1.18092E+1 1 1.10024E+11 1.09898E+1 1 

ON 2.0733 1E+11 2.22043E+1 1 2.07265E+11 2.07495E+11 

MA 16,146,993,217 17,504,024,442 16,701,823,718 16,641,840,086 

SK 13,982,423,360 15,129,217,320 14,311,467,435 14,255,519,224 

AB 52,594,490,290 57,359,188,114 53,195,227,508 53,077,388,907 

BC 56,206,787,033 59,886,429,369 56,875,663,895 56,764,297,512 

National 4.85784E+11 5.23184E+11 4.911 16E+11 4.90763E+11 

Table 6 

Comparison of relative errors in estimating income before taxes (%) 

Province GWSM Method (1) Method (1) Method (2) Method (2) 

with missing links using mean using median using mean using median 

NFL 5.688 0.599 0.460 1.059 2.397 

PEI 5.075 0.570 0.532 2.859 4.063 

NS 7.094 0.749 1.148 3.549 2.459 

NB 6.433 0.037 0.397 2.693 2.987 

QC 7.492 0.828 0.909 4.372 2.896 

ON 6.267 1.413 1.348 4.691 a7 

MA 6.856 0.838 0.500 1.644 3.654 

SK 6.988 0.107 0.446 2.480 2.598 

AB 7.982 0.502 0.696 3.185 2.407 

BC 5.926 0.442 0.612 3.995 3.343 

National 6.734 0.650 0.724 3.868 2.662 
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Table 7 
Comparison of relative errors in estimating income after taxes (%) 
Province GWSM Method (1) Method (1) Method (2) Method (2) 
with missing links using mean using median using mean using median 
NFL 5.617 0.588 0.457 1.101 2.409 
PEI 5.061 0.616 0.585 2.832 4.121 
NS T2219 0.491 0.887 3.338 2.765 
NB 6.522 0.127 0.226 Pepe, 3.150 
QC 7.742 0.403 0.496 3.991 BEN) 
ON 6.387 1.130 1.068 4.432 2.081 
MA 6.836 0.881 0.551 1.645 3.133 
SK 6.977 0.027 0.356 2.406 2.675 
AB 7.984 0.493 0.684 3.180 2.415 
BC 5.909 0.406 0.584 3.989 3.419 
National 6.841 0.415 0.492 3.657 2.927 
Table 8 
Comparison of relative errors in estimating earnings (%) 
Province GWSM Method (1) Method (1) Method (2) Method (2) 
with missing links using mean using median using mean using median 
NFL 6.286 1.682 1.509 0.041 3.585 
PEI 6.382 3.470 3397, 0.0739 PAIS 
NS 8.115 1.773 1.308 1.265 5.281 
NB 6.930 1.430 1.062 1.279 4.512 
QC 8.472 O773 0.706 2.827 4.560 
ON 6.955 0.336 0.213 3.760 2.920 
MA 8.172 2.879 2oN2 0.291 5.835 
SK TADS 1.467 1.055 0.979 4.324 
AB 8.739 0.691 0.475 2.140 Sm TIE 
BC 6.553 1.091 0.914 2.643 5.081 
National aden 0.715 0.664 2.628 4.131 


They show that our estimates using both method (1) and 
method (2) perform very well in terms of reducing bias. 
Method (1) does work better than Method (2) overall, yet 
the improvement from Method (1) to Method (2) is much 
less compared to that made by moving from without 
adjustment to method (2). Since Method (2) provides us 
with high quality and involves much less information than 
Method (1), Method (2) is recommended. 

Now, we focus on Method (2) using mean, which gives 
the estimate Y2), to analyze how its variance performs in 
terms of estimating Y°. We use the bootstrap technique to 
estimate the variance of Y2) at both the national level and 
the provincial level. The bootstrap used for our simulation in 
this paper is the classical Bootstrap with replacement, where 
bootstrapping is performed at the first stage of sampling. 
The bootstrap weights taken here are provided with the 
SLID data, and incorporate all the necessary adjustments. 
See Lévesque (2001), and LaRoche (2003) for details on the 
use of the Bootstrap for SLID. The improvement in 
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reducing the variance is not as large as in reducing bias; 
however, it is revealed in this simulation study that the 
proposed method provides a smaller variance as well 
compared to applying GWSM without an adjustment for 
missing links. See Table 10 for the results. 

The simulation results presented here are based on a 
single sample of SLID and a single random removal of the 
links of initially-present individuals. For a complete 
assessment of the properties of the above estimators, a 
Monte-Carlo process would have been suitable. Such 
simulations have been performed by Hurand (2006) based 
on agricultural data. In these simulations, 1,000 samples 
have been selected and for each selected sample, the worst- 
case-scenario has been used, ie., all links from the non- 
sample units have been removed. The results of these 
simulations showed that proportional adjustment and global 
proportional adjustment are the two methods whose 
estimates are, on average, the closest to the real total, and 
whose biases are negligible. 
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Table 9 
Comparison of relative errors in estimating wages and salaries before deductions (%) 
Province GWSM Method (1) Method (1) Method (2) Method (2) 
with missing links using mean using median using mean using median 
NFL 6.336 1.656 1.484 0.1012 3.593 
PEI 6.809 4.734 4.694 1.056 8.424 
NS 8.231 2.047 1.573 0.939 5.509 
NB EAS) 2.036 1.664 0.685 5.133 
QC 8.704 1.278 1.162 2.294 5.070 
ON 7.096 0.0317 0.0791 3.473 3.265 
MA 8.404 3.436 3.065 0.787 6.469 
SK 8.202 2953 1.953 0.107 S213 
AB 9.059 1.142 0.918 8) 4.247 
BC 6.547 1.190 0.992 2.565 5.234 
National 7.699 1.098 1.025 2.291 4.54] 
Table 10 
Comparison of standard deviation estimates 
Variables Total income Total income Earnings Wages and salaries 
before taxes after taxes before deductions 
National GWSM with missing links 9,677,258,789 7,343,792,762 8,850,202,075 8,468,718,449 
level Method (2) using mean 9,471,103,083 1238 Mis o23 8,593,015,854 8,232,428 642 
Ontario GWSM with missing links 7,888, 106,377 6,101,001,739 7,245 ,688,373 7,149,203,530 
Method (2) using mean 7,601,169,501 5,939,509,894 6952217812 6,83 1,300,511 
Quebec GWSM with missing links 4,341,215,711 3,113,247,130 3,772,369,180 3,162,277,660 
Method (2) using mean 4,160,251,472 2,974,248,451 3,668,996,929 3,100,868,366 
5. Closing remarks LUT PUR Oe A IG Bae BP 
16g 


We have constructed four estimation methods to address 
the link nonresponse problem in indirect sampling. The 
simulation results in this article show that the adjustments 
methods we have presented in the example for using 
GWSM incorporating the link nonresponse performs well in 
terms of both reducing the estimation bias and providing an 
overall improvement in variance. The advancement in bias 
reduction seems significant. The implementation of the 
methods proposed in Section 3.2 for real data sets will be 
studied in the near future. 

The following significant observations emerged from our 
study: 


1. Adjustment methods are simple to apply. 


2. In a more general situation, such as L,, >1 for 
some j’s, (35) represents the weighted mean 
weighted by Lig Accordingly the median 
approach delivered by (39) and (40) can be 
modified using a generalized version of median — 
“weighted” median. Namely, we replace (38) by 


: . 1 
= the median of — 
ei 


3. Some valid link responses outside s* can not be 
used in estimating L? by the methods proposed in 
Section 3.1. However, this valid information would 
be beneficial to the approaches by predicting /, , 
using auxiliary variables, as can be seen in 
Section 3.2.1. 
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Nonparametric propensity weighting for survey 
nonresponse through local polynomial regression 


Damiao N. da Silva and Jean D. Opsomer ' 


Abstract 


Propensity weighting is a procedure to adjust for unit nonresponse in surveys. A form of implementing this procedure 
consists of dividing the sampling weights by estimates of the probabilities that the sampled units respond to the survey. 
Typically, these estimates are obtained by fitting parametric models, such as logistic regression. The resulting adjusted 
estimators may become biased when the specified parametric models are incorrect. To avoid misspecifying such a model, 
we consider nonparametric estimation of the response probabilities by local polynomial regression. We study the asymptotic 
properties of the resulting estimator under quasi-randomization. The practical behavior of the proposed nonresponse 


adjustment approach is evaluated on NHANES data. 


Key Words: Kernel regression; Missing data; Propensity scores; Unit nonresponse; Weighting adjustment. 


1. Introduction 


Propensity weighting is a procedure that is often applied 
in sampling surveys to compensate for unit nonresponse. 
Under this type of nonresponse, complete data collection is 
accomplished at only a part of the units selected to the 
sample, which are termed as the respondents. The 
propensity weighting procedure operates by increasing the 
sampling weights of the respondents in the sample using 
estimates of the probabilities that they responded to the 
survey. These probabilities are also referred to as response 
propensities in virtue of their analogy with the propensity 
score theory of Rosenbaum and Rubin (1983) for 
observational studies, incorporated into survey nonresponse 
problems by David, Little, Samuhel and Triest (1983). 

General descriptions of propensity weighting to adjust 
classical survey estimators for nonresponse can be seen, for 
example, in Nargundkar and Joshi (1975), Cassel, Sarndal 
and Wretman (1983) and Groves, Dillman, Eltinge and Little 
(2002). Traditionally, the way the procedure is implemented 
estimates the response probabilities with parametric regres- 
sion curves, such as logistic, probit or exponential models. 
See Alho (1990), Folsom (1991), Ekholm and Laaksonen 
(1991) and Iannacchione, Milne and Folsom (1991) for 
earlier references. A recent theoretical account of the 
statistical properties of the procedure is given in Kim and 
Kim (2007). These parametric models are readily fitted as 
generalized linear models. However, an important and 
sometimes overlooked part of this procedure is the specifica- 
tion of the form of the link function to relate the response 
propensities and a linear predictor of the auxiliary informa- 
tion. If this function, which we shall refer to as the response 
propensity function, is misspecified, the resulting adjusted 
estimators of the population quantities are likely to be biased. 


Another approach to estimate the response propensities is 
through nonparametric methods. The main motivation to 
use such methods is that the parametric form for the 
response propensity function need not be specified. In this 
sense, these methods offer an appealing alternative to the 
choice of a link function, as raised by Laaksonen (2006), or 
when a parametric model is difficult to specify a priori. In 
this context, Giommi (1984) proposed using kemel 
smoothing, in the form of the Nadaraya-Watson estimator, 
to estimate the response probabilities. Da Silva and 
Opsomer (2006) established the consistency of Giommi’s 
estimator for the population mean and derived rates for the 
asymptotic bias and the variance. Theoretical properties of a 
Jackknife variance estimator were also studied. 

In this article, we extend the results of Da Silva and 
Opsomer (2006) in two directions. First, we consider the 
estimation of the response propensities by local polynomial 
regression, a nonparametric technique described, for 
instance, in Wand and Jones (1995). Compared to kernel 
smoothing, local polynomial regression improves the local 
approximation to the unknown propensity function, which 
results in better practical and theoretical properties. It is also 
much more prevalent as a smoothing method in practice, 
with implementations available in most major statistical 
programs. Second, we apply the nonparametric propensity 
score estimation approach to data from the National Health 
and Nutrition Examination Survey (NHANES), which 
makes it possible to compare several nonresponse 
adjustment methods, both parametric and nonparametric, in 
a realistic setting. 

In Section 2, we introduce the weighting procedure and 
the estimation of the response propensities. The theoretical 
properties of the adjusted estimators are discussed in Section 
3. In section 4, we describe how to adapt a replication 
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variance procedure to estimate the variance of the proposed 
adjusted estimators. Finally, in Section 5, we demonstrate 
the finite sample properties of the estimators by means of a 
simulation experiment using data from NHANES. 


2. Weighting by local polynomial regression 


Consider a population of N, units, denoted by U, = 
{l, 2, .... N,}. Suppose that a sample s, is drawn from 
U,, according to some probabilistic sampling design 
PAS): “Leth 7 2 betsthe Sizer Of) Siar ANG Tujhe. Tape 
Pr{i € s,} = Dy: ies, P(S,) be the inclusion probability of 
unit i, for all ie U,. It is of interest to estimate the 
PoRuatons mean of a study variable y, namely 
Vrs pe y;, where y, denotes the value of y for 
the i" unit of U,. We assume that the values x, of an 
auxiliary variable x are fully observed throughout the 
sample. Let y, = ();,---. Yy,), and similarly for x, 

When the sample contains unit nonresponse, we only 
observe the values of the study variables for the units in a 
subset r, C s,. To account for the information lost in the 
estimation of the parameters of interest, it becomes 
necessary to model the response process. To define this 
response model, let R, be an indicator variable assuming 
the value one if the unit 7 respond to the survey, and the 
value zero otherwise, for all i € s,. We assume that, given 
the sample, the response indicators are independent 
Bernouilli random variables with 


Pr{R, =1lies,, y,,xv} = (x) =, for all ie s,, (1) 


where the exact form of the response propensity function 
o(-) is unspecified, but it is assumed to be a smooth 
function of x, with (-) € (0, 1]. The relationship in (1) 
defines a nonresponse process said to be ignorable, in the 
sense that the response propensities are independent of the 
values of any study variable, conditional on the covariate x 
(see Lohr 1999, page 265). The theory developed here, 
therefore, does not intend to handle non-ignorable response 
mechanisms. 

If all response propensities were known, resulting 
weighting adjustments could be obtained by applying a two- 
phase estimation approach. For instance, two possible 
estimators of the population mean y y, would be given by 


Ynyv = — Day 2) 


v ies, 


and 


y rat, Tov 


a ee ae ee (3) 


ies iés, 
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which are forms of adjustments for the Horvitz-Thompson 
and the Hajek estimators to compensate for the unit non- 
response. The same ideas can be used to obtain propensity 
weighting adjustments for the generalized regression esti- 
mator for estimation in the presence of nonresponse (Cassel 
et al. 1983). 

Estimators (2) and (3) are unbiased and nearly unbiased 
for Vy respectively, under the quasi-randomization 
approach of Oh and Scheuren (1983), where the statistical 
properties are evaluated using the joint distribution of the 
sampling design and the response model. However, the 
response propensities are usually unknown in practice and 
we need to replace the 9, in (2) and (3) by estimates b,, 
satisfying 0 < b, <1. The resulting propensity weighting 
estimators are therefore 


Vay = Dy Oo WR, (4) 
and 


= Oey: 


1ESy 


y rat, tov 


TES. 


iol aeartags (5) 


The latter formula has the advantage of being location-scale 
invariant, because the summation of its adjusted weights 
TC, 6; ness TE, b;" R, is equal to one, and does not 
require the population size N,, to be known. 

In order to implement the propensity weighting esti- 
mators (4) and (5), it is necessary to estimate the response 
propensities b,. Da Silva and Opsomer (2006) used kernel 
regression for this purpose. The procedure we consider here 
is local polynomial regression, which can be described as 
follows. Let K(-) be a continuous and positive kermel 
function and h, be its bandwidth. Define the N, x (k + 1) 
matrix 


ae) (6 A 


Xy; = 


k 
(Xy, — x;) | 


1 2 ame, 
— K|— ol SUjasoN ie 
h, h, 


and population vector of response indicators Ry = 
(R,, R,,.... Ry) The vector R, would be known if, 
instead of the sample s,, a census was considered from the 
population U,,. In that case, the local polynomial regression 
estimator of degree k of , = (x,), based on the whole 
population, would be given by the fit 


bes @iylt93}) 


the NV, x N, matrix 


W,; = diag 
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bu; = 61 (Xt; Wor Xu) Xt Wui Ro, (6) 
where e, denotes the 7" column of the identity matrix of 
order k +1 and it is assumed that X,, W,, X,; 
singular. 

Since the values of the response indicators are only 
observed for those units selected into the sample, the 
population fit (6) is unfeasible. However, defining X,, as 
the n, x(k +1) matrix formed with the rows of X,, 
corresponding to the units 7 € s,, 


K eMail : JES 
tt, h, h, y 


and R, =(R;; j€s,)', then a sample-based local 
polynomial regression estimator of degree k of o, = 0(x;,) 
is given by 


iS non- 


W,; = ig 


~ A 


0; =, e iB t,; (7) 


where 


A 


( si? oe) = (7 Si, Ene oe ==]9 Caged 
= (X,, W,, X,;, X;, W, R,) 


si? 


and it is assumed that T,, is invertible. An special case of 
(7) is obtained by considering k = 0, which corresponds to 
the kernel regression estimator of Da Silva and Opsomer 
(2006). Other special cases from (7) are the local linear, the 
local quadratic and the local cubic response propensity 
estimators, which result from the local fit of polynomials of 
degree one, two and three, respectively. 

In practice, when be happens to be singular, a simple 
procedure to insure that o? is well defined is choosing a 
bandwidth large enough to guarantee at least k + 1 values 
of R, in the window [x, —A,, x,+h,], for allie s,. If 
this window does not contain enough responses indicators 
and the bandwidth has to remain fixed, another approach 
has to be considered. To this purpose, we adopt here the 
adjustment made by Breidt and Opsomer (2000) and define 
the sample-based local polynomial regression estimator of 
degree k of , = o(x,) by 


| 
fon bh) =e[f, edie Se} tres. © 


where 6, is some small positive constant. The smaller order 
terms 6,/N, added to the main diagonal of T,, are 
sufficient to make the resulting adjusted matrix invertible 
for any h,. As a consequence, (x, k, h,) will be well 
defined, for all ice s,. However, another technical 
difficulty to use (x, k, h,) as a propensity weighting 
adjustment arises because the response propensity estimator 
(8) can indeed become arbitrarily close to zero. To tackle 
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this problem, we bound (x, k, h,) away from zero by 
considering the estimator 


b, = max {O(x, k, h,), 5(N,h,)'}, (9) 


for some constant 5, >0. This idea is related to the 
adjustment made by Da Silva and Opsomer (2006) for the 
kernel regression estimator. 


3. Asymptotic properties 


In this section, we present the properties of the propensity 
weighting estimators (4) and (5) under estimation of the 
response propensities by the local polynomial estimator (9). 
The assumptions, lemmas and outlines of the proofs for the 
following results are given in the Appendix, and a complete 
theoretical investigation can be found in Da Silva and 
Opsomer (2008). The full derivations are not reported in this 
article, because they follow the general approach described 
in DaSilva and Opsomer (2006). We consider an 
asymptotic framework by which the population U, is 
embedded into the increasing sequence of populations 
(U.N GN a) er tomnench Ua a saluple=s. 01 size 
n,(n, 2 n,_,) 1s selected according to a sampling design 
p,(-). This framework is commonly adopted in asymptotic 
studies of survey estimators. See Isaki and Fuller (1982) for 
an early reference. 

As a population-based approximation for 9, = (x;), we 
shall consider in the derivation of most results in this section 
the population fit by local polynomial regression 


b= Oe, 6h) =e.B, =e. t, LeU. . (0) 
where 
=* k+l] k+l 
Cr: t,) oa ({ a ip) pe i) 
= E(T,, ae a (X;; Wo ; Xyjip xX), W, Wi oy), 


the matrices X,, and W,, are as in (6) and 6, = 
(6(%,), O(%), ---» O(%y,)). The following theorem states the 
asymptotic properties of Y.;, under a set of assumptions in 
the Appendix. These assumptions are regularity conditions 
on the sampling design and the finite population, both of 
which are standard infinite population asymptotics, 
ignorability conditions on the nonresponse mechanism, and 
a set of standard regularity conditions related to the local 
polynomial regression of the response propensity function. 


Theorem \. Assume the assumptions (A1)-(A4), (B1)-(B3) 
and (Cl)-(C5) in the Appendix hold. Consider the 
estimation of the population mean y, by the propensity 
weighting estimator Y 4, defined in (4), and suppose the 
response propensities are estimated by b,, the local 
polynomial regression estimator of degree k in (9). Let 
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= wiy 
1; a4; Rs 


N, ies, 


ein at (11) 


where 
Wi; = b;" - b, e; (t,, "7 T,, B,), 


and T,, are given in (7) and $,, B,, T, are defined in 


si 


(10). Then, 
Vaal cle : 12 
El@ eon nn lie Be (12) 
and the bias and variance of Y,., Satisfy 
B, = ELY nv at Yn] 
o[ Ate?) + of ) k even, 
n, h, 
= (13) 
ofa) of k odd, 
ny, h, 
and 
Vat yee. ene (14) 
pr ny h, 


Results (12) and (13) imply that the propensity weighting 
estimator ),3,, using a response propensity estimator based 
on local polynomial regression, is asymptotically unbiased 
for the population mean yy under the joint distribution of 
the sampling design and the response model (1). Combining 
this result with (14), then we obtain that 


Z S. 
Vay IN +0[s4-] (15) 
nN, h, 
when the bandwidth satisfies 
1 
als eT, ) k even, 
ees (16) 


] 
aly 2k3 ) k odd. 


Hence, without assuming a parametric form for the response 
propensity function $(-), yg, is consistent for the popula- 
tion mean with respect to the sampling design and the 
response model, as long as the response propensities are a 
smooth function of the covariate x. As a price paid for this 
robustness, the rate of convergence is of order ./n, h, 

instead of the usual parametric rate Jn, . However, as the 
degree of the local polynomial k increases, the rate of 
convergence improves. Since the kermel regression esti- 
mator in Da Silva and Opsomer (2006) is equivalent to the 
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case k = 0, local polynomial regression with higher degree 
is asymptotically superior to kernel regression in the context 
of a nonresponse adjustment. This theoretical finding is 
consistent with that in other contexts (see e.g., Wand and 
Jones 1995, page 130). 

Expression (11) on Theorem 1 generalizes another 
finding from Da Silva and Opsomer (2006) to the case of 
local ee atone regression, which is that the asymptotic 
weights \y,' cannot be eagiy cue by the inverse of 
response gS hg ,' (or their population-level esti- 
mators o; ). One immediate consequence is that the esti- 
mator ) ,3, is not asymptotically equivalent to y ,,, in (2). 

The following corollary provides an asymptotic distribu- 
tion for y ,g,, assuming the asymptotic normality of Y _,,. 
Corollary 1. Assume the conditions of Theorem \ hold. 
Suppose that the sampling design and the response model 
are such that 


Y xv an Vy, _ B, 
[Var(¥ pg )) 

where B., is defined in (13). If additionally 

lim (n, h,) Var(V.g,) € (0, ©), 


IN (On asin ce 


then 

= PEST fa + N(O, I). 
[Var(y gv] 

We now discuss the properties of the ratio-based version 
of propensity weighting estimator given in (5). Based on the 
results for );,, standard ratio estimation theory can be 
used to derive asymptotic results for Y,,, .3,- In particular, 
under the same assumptions the asymptotic rates for the 
approximate bias and variance of y,,,,3, are the same as 
those in Theorem 1, and the asymptotic distribution of 


Vite is given in the following result. 


Theorem 2. Assume the conditions of Theorem \ hold. 
Suppose the population mean is to be estimated by the 
propensity weighted estimator Y.. x.éy of (5) and the 
response propensities are estimated by b,, the local 
polynomial regression estimator of degree k defined in (8). 
Let 


= Kees 
Bon = 5 LBW, Fad 


where the weights \y,' 
that 


are given in Theorem |. Suppose 


E(ex. ) 
peeraaar ste l) aav>o, 
TV 
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and 


lim (n, h,)Var(@gy) € (0, ©). 


Then, 
Vena ws Yn. =e Jet ® 
[Var(@,y)} 


as v->0, where B,,, = O(hi"), if k is odd, and 
Bay = O°), if k is even. 


BEES SOP) 


4. Variance estimation 


As noted in Section 3, the estimator 4, is not 
asymptotically equivalent to y,,,, so that approximating 
the asymptotic variance of the former by that of the latter is 
typically incorrect. In fact, a proof that the asymptotic 
variance of y,,, overestimates the variance of y eS 
given by Kim and Kim (2007) when the response 
propensities are assumed to follow a parametric model. In 
the present context, the asymptotic variance of y 


Dé v'R y, 


N, IES, 


nv | 


9 


Var[ Y nv = Var| — 


with Wy,’ given in Theorem 1. As was previously noted in 
Da Silva and Opsomer (2006) for the simpler case of a zero 
degree polynomial, the high level of complexity in the 
expression makes direct estimation of this variance 
impractical, and a replication method was proposed instead. 
We briefly outline the procedure here, which is extended to 
local polynomials of degree k. We omit the theoretical 
derivations. 

We start from a set of replicate weights in the absence of 
nonresponse, defined for estimating the variance of a linear 
estimator 


6-—-¥ wy. 
IN vies, 


The replicate variance estimator for 6 is defined as 
V(6) = 34 apg 16) (17) 
where 
Spin LoS Wee pee eT 


v ies, 


denotes a set of L, replicates for 6, w\ are sampling 
weights associated with the ¢™ replicate and c, is factor 
that depends on the replication procedure. Examples of 
replication procedures satisfying (17) use variants of the 
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Jackknife method or the Balanced Repeated Replication 
technique. The process to adapt the replication procedure to 
estimating the variance of Vrov Ad Yiu ngy 1S straight- 
forward. The needed replicates of these adjusted estimators, 
namely ate and Ye tes , are obtained by replacing the 
Ww, = i by wo in (4) and (5), respectively, and also in 
the computations needed to produce the b, in (9). In section 
5.4 below, we evaluate the practical performance of the 
replication variance procedure on NHANES data. 


5. Application to NHANES data 


5.1 The NHANES design 


We evaluate the performance of the local polynomial 
adjusted estimators on real data. We consider the 2005-2006 
release of the National Health and Nutrition Examination 
Survey (NHANES), which is conducted by the National 
Center for Health Statistics, Centers for Disease Control and 
Prevention (NCHS/CDC), of the U.S. Department of Health 
and Human Services. This survey consists of a stratified, 
multistage sample of the U.S. civilian non-institutionalized 
population. A general overview of the sample formation is 
as follows: 


(i) within each stratum, primary sampling units 
(PSUs) consisting of counties or grouped smaller 
counties are selected by sampling with probabilities 
proportional to a measure of size; 

(ii) from the sampled PSUs, groups of city blocks 
(segments) containing clusters of households are 
selected also by sampling with probability 
proportional to size; 

(iii) in the selected segments, clusters of households are 
randomly selected with varying _ selection 
probabilities to oversample groups of age, ethnic, 
or income in certain geographic areas; and 

(iv) in the selected households, one or more participants 
are selected randomly. 


The public release of NHANES data has two important 
aspects. First, to reduce disclosure risks, the stratified, four- 
stage survey is condensed in a stratified one-stage design, 
with neither the new stratum variable nor the new PSU 
variable corresponding to the same variables in the original 
design. Secondly, the base sampling weights, obtained by 
reciprocal of the inclusion probabilities of the survey 
participants, are not released. The weights provided reflect 
adjustments made to the base weights to account for unit 
nonresponse, in the interview and exam portions of the 
survey, and to produce estimates satisfying known 
population controls. 
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5.2 The simulation experiment 


In order to empirically evaluate the local polynomial 
estimators as adjustments for nonresponse in complex 
surveys, we will apply an artificially generated source of 
unit nonresponse to the public-release NHANES dataset. 
The nonresponse mechanism will be taken as a smooth 
function of the age in years of the survey participant (AGE). 
For this comparison, we chose as study variables four 
characteristics related to heart diseases, namely the systolic 
blood pressure (SBP), the diastolic blood pressure (DBP), 
the indicator of hypertension (HTN) and the indicator of 
high serum total cholesterol (HTC). All of these were 
measured on survey participants who were 18 years or 
older. The systolic and diastolic variables were obtained as 
the average of the corresponding measurements in a set of 
up to four readings. Hypertension was defined for 
individuals having systolic blood pressure of 140 mm Hg or 
higher or a mean diastolic blood pressure of 90 mm Hg or 
higher or currently taking medication to lower high blood 
pressure. High serum total cholesterol was considered when 
the individual had a total serum cholesterol greater than or 
equal to 240 mg/dL. The unweighted sample correlations 
among these and the AGE variable are 0.481 (SBP), 0.118 
(DBP), 0.552 (HTN) and 0.060 (HTC), respectively. Hence, 
it is reasonable to postulate that unit nonresponse related to 
age is likely to have different effects on survey estimators 
for these four variables. 

The total number of eligible individuals in the NHANES 
dataset is 4,727. We generated unit nonresponse for the four 
variables of interest according to two logistic response 
propensity functions of the auxiliary variable x taken by the 
age (in years) of the survey participant minus 18. These 
functions consider a linear and a nonlinear predictor of x as 
follows 


Linear predictor: 


,(x) = {1 + exp[-(By + B,x)}}" 


Nonlinear predictor: 


1 (x) = 
(1+ exp[-(B, +B,x+B,x7+, cos(B,x?/n) sin(B,x/z))]} 


where the regression coefficients Bo, ...,; were chosen so 
that the response propensity functions give an overall 
nonresponse rate of about 30% when applied to the sample 
values of x. In both cases, we kept the NHANES sample 
fixed and generated B=1,000 independent response 
indicator vectors by Poisson sampling. 

The following six nonresponse adjustments were evalu- 
ated on these data. Note that in all cases we reported the 
ratio versions (5) of the estimators, because they were found 
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to be much more precise than the Horvitz-Thompson 
versions. 


1. True response probabilities: 6, = o(x,), 1 € S,. 

2. Logistic regression adjustment: 6, obtained as the 
estimated probabilities from a logistic regression of 
each response vector on x, using a polynomial in x 
of degree one as the linear predictor. 

3. Weighted local polynomial regression of degree k 
and bandwidth h,: 6, = 6(x, k, h,) given by (8), 
with ie s,, k =0,1,2,3, h, =0.15, 0.25, 0.50 and 
the Epanechnikov kemel function 


K(t) = G/4)(1 — x”) 14x] < lh. 


4. Unweighted local polynomial regression of degree k 
and bandwidth h,: the same as above but not 
including the sampling weights in (8) to obtain the 
6, = = (x,, k, h,). This might be somewhat easier to 
compute in practice and should lead to similar results, 
even if it does not, strictly speaking, follow the 
pseudo-randomization theory of Section 3. 

5. Weighting within cell: within each stratum, respon- 
dents and nonrespondents were classified into four 
classes of age based on the sample quartiles of this 
variable. This procedure subdivided the sample in a 
total of 60 cells. Let s, and s,, denote respectively 
the set of sampled elements and the set of responding 
elements in the g" cell. Then, the WC adjustment is 
defined by taking 


for all respondents Les. 
6. Naive: 6, =lies,. 


5.3 Bias and robustness against a misspecified 
response propensity function 


When the full sample without artificial nonresponse is 
used, the Hajek estimated means for the four study variables 
are respectively SBP = 122.19 mmHg, DBP=70.29 mm Hg, 
HTN = 29.04% and HTC=15.76%. Table 1 gives the 
percentage bias relative to those means across response sets 
obtained for every adjustment procedure in this simulation 
experiment. For both weighted and unweighted Local 
Polynomial Regression adjusted estimators, we only display 
the results for the bandwidth h, = 0.25, but those for other 
bandwidth values are similar. We instead show the results 
for different degrees of the local polynomial, so that the 
effect of moving from local constant to higher order 
polynomials can be evaluated. 
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Table 1 

Relative biases (%) of nonresponse-adjusted estimators for mean systolic blood pressure (SBP), diastolic blood pressure (DBP), 
indicator of hypertension (HTN) and indicator of high serum total cholesterol (HTC), based on 1,000 response sets for two 
propensity functions of the age of the survey participant in NHANES 2005-2006 


Logistic propensity function Logistic propensity function 


(linear predictor) (nonlinear predictor) 
Type of adjustment SBP DBP HTN HTC SBP DBP HTN HTC 
True Response Propensities 0.01 0.01 -0.01 0.04 -0.00 -0.00 0.01 -0.22 
Logistic Regression 0.01 0.00 -0.03 0.03 0.47 -1.67 6.49 -6.76 
Weighted Local Polynomial 
Regression: 
Degree 0 0.27 0.34 3.39 2.41 -0.20 -0.39 -1.20 -2.27 
Degree | 0.00 0.04 -0.03 0.20 -0.01 -0.49 0.34 -2.36 
Degree 2 0.01 0.01 0.03 0.07 0.03 -0.05 0.51 -0.27 
Degree 3 0.01 0.01 -0.02 0.04 -0.03 -0.05 -0.24 -0.44 
Unweighted Local Polynomial 
Regression: 
Degree 0 0.11 0.24 1.34 153 -0.17 -0.47 -0.98 -2.70 
Degree 1 0.01 0.05 -0.00 0.25 -0.01 -0.57 0.34 -2.69 
Degree 2 0.01 0.01 -0.00 0.07 0.01 -0.07 0.26 -0.40 
Degree 3 0.00 0.01 -0.06 0.03 -0.03 -0.06 -0.29 -0.48 
Weighting Within Cell 0.08 0.08 0.84 0.69 -0.11 -0.07 -0.84 -0.48 
Naive 1.62 0.80 20.49 8.04 -1.30 -1.60 -15.61 -10.77 


Among the estimators affected by the generated 
nonresponse, the worst bias performances are clearly for the 
unadjusted “Naive” estimator. As displayed in the last row of 
Table 1, the biases are higher in the estimation of the 
prevalence of hypertension and the mean systolic blood 
pressure, as these are the characteristics of the study 
variables with higher correlations with the AGE variable, 
and also for the prevalence of high serum total cholesterol. 
The biases of the Naive estimator can be successfully 
reduced with the true response propensity estimator, any of 
the local polynomial regression adjusted estimators, the 
weighting-within cell estimator or with the logistic adjusted 
estimator, if the model for the propensity function is 
correctly specified. The best performances in terms of small 
bias are obtained using the estimator adjusted by the true 
response propensities, because it is conditionally unbiased 
for the full sample estimates. The logistic adjustment when it 
is applied under the correct model, given by the propensity 
function with a linear predictor, also gives nearly unbiased 
estimates. For the second propensity function, where the 
form of the predictor is not well captured by the logistic 
regression fit of a regression line, this adjustment yields a 
conditionally biased estimator. 

The averages of the local polynomial regression estimates 
become generally closer to the full sample estimates by 
increasing the degree of the polynomial fitted, with the 
largest jump when moving from a local constant to a local 
linear estimator. Hence, it seems that local polynomial 
regression is indeed superior to kernel regression in this 
context. There is very little difference between the weighted 
and unweighted forms of this adjustment and both 
procedures have overall smaller conditional biases than the 
biases of the weighting-within cell estimator, when they are 
implemented by fitting locally a polynomial of order greater 


than zero to estimate the response propensities. The zero 
degree propensity weighted and unweighted adjusted 
estimators have smaller biases at smaller bandwidths, as we 
observed with the bandwidth 0.15, for instance, but smaller 
bandwidths tend to increase the variance of the estimators. 
Overall, both weighted and unweighted local polynomial 
regression adjustments outperform the parametric logistic 
adjustment when the response model is misspecified. By 
implementing the local polynomial adjustments with degrees 
above one, their performances are similar to the one of the 
logistic adjustment under the correct specification of the 
response model. 


5.4 Variance and variance estimation 


Table 2 shows the variance of the adjustment methods 
considered here across the nonresponse replicates, and we 
normalized them by the variance for the true response 
propensity adjustment for clarity. Interestingly, there appears 
to be an inverse relationship between the magnitude of the 
relative biases in Table 1 and the variances in this table. In 
those cases where the relative bias was small (the weighted 
and unweighted local polynomial regression, the weighting 
within cell as well as the logistic regression adjustment for 
the linear propensity function), all the methods appear to 
result in roughly similar variances. There is a tendency for 
higher degree local polynomials to be more variable than 
lower degree ones, and this is particularly noticeable for the 
nonlinear propensity function, where a clear jump is seen 
when one moves from degree 1 (local linear) to 2 (local 
quadratic). Overall, it seems that local linear regression, 
either weighted or unweighted, offers a good compromise 
between the bias and the variance of the nonresponse 
adjustment procedure. 
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Table 2 
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Normalized Monte Carlo variances of nonresponse-adjusted estimators for mean systolic blood pressure (SBP), diastolic blood 
pressure (DBP), indicator of hypertension (HTN) and indicator of high serum total cholesterol (HTC), based on 1,000 response sets 
for two propensity functions of the age of the survey participant in NHANES 2005-2006 


Logistic propensity function 


Logistic propensity function 


(linear predictor) (nonlinear predictor) 
Type of adjustment SBP DBP HTN HTC SBP DBP HTN HTC 
True Response Propensities 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 
Logistic Regression 85.9 92.4 125) 96.5 63.9 61.5 54.1 52.0 
Weighted Local Polynomial 
Regression: 
Degree 0 74.9 81.2 65.7 ol 70.3 67.0 67.4 75.0 
Degree | 81.8 89.5 66.2 92.7 73.6 69.8 68.9 76.0 
Degree 2 81.3 89.8 65.5 94.0 90.3 81.7 88.0 96.1 
Degree 3 82.3 90.2 65.8 O3nl 90.1 82.2 87.7 96.2 
Unweighted Local Polynomial 
Regression: 
Degree 0 82.2 85.8 77.6 95.8 Tee) 69.2 70.7 74.7 
Degree | 85.6 90.1 79.4 O57) 74.4 71.1 Ie 74.6 
Degree 2 86.6 De 7933 96.1 91.8 84.5 91.8 96.8 
Degree 3 87.3 DS 78.5 95.0 O12 84.7 Na) 96.9 
Weighting Within Cell TEE 89.1 62.1 91.6 82.5 77.0 81.1 92.3 
Naive es 58.0 81.7 74.6 48.6 48.7 45.5 45.1 


The above simulation results showed the behavior of 
several nonresponse adjustments in the NHANES setting. 
We now consider the replication variance estimation 
approach of Section 4 and evaluate its usefulness as a 
sample-based measure of uncertainty for the nonresponse- 
adjusted estimators in the same setting. We implemented 
(17) with the Jackknife method. Since NHANES does not 
provide information on the joint sample inclusion 
probabilities, we could not apply a full Jackknife variance 
estimator as in, for instance, Berger and Skinner (2005), as a 
means to account for the selection of units with varying 
probabilities in the survey. Because of this, we assumed the 
within-stratum designs in NHANES could be approximated 
by cluster sampling with replacement and rewrite (17) in the 
form proposed by Rust (1985), 


ih 
AO 2 Cy ("= 6), 


JES, 


(18) 


where s, denote the set of units in sample from the ¢” 
NHANES stratum, ¢ = 1, 2, .... T, n, be the number of 
units selected to s,, c, = (n, —1)/n, and 6” is obtained 
from (5) by replacing the w, with the replication weights 


0, for a survey participant 
ie PSU JZ jes, 
Wi) = \n,/(n, —1)w,, for a survey participant 


iePSU /, j'es,/(j 49) 


W,, for a survey participant 
i€S,. 
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These weights were also applied in the estimation of the 
response propensities for the weighted local polynomial 
regression procedure adjustment procedure. 

The Jackknife variance estimator (18) was applied to 
each response vector from the two propensity functions, 
yielding estimates ?,,(0(b)), 6 =1, 2,...,B, for all 
adjusted estimators in the experiment. For the sake of 
comparison, it would be informative to produce estimates of 
the corresponding variances by the Monte Carlo method. 
However, as the NHANES sample is fixed, the Monte Carlo 
variance of the point estimates 6(b) across response 
vectors estimates only the conditional variance Var(6|s,) 
with respect to the response model. Since 


Var(6) = Var(E(6|s,)) + E(Var(6|s,)), 


where the “inner” moments are taken with respect to the 
response model given the sample and the “outer” moments 
are with respect to the sampling design, the design variance 
of E(8|s,) needs to be accounted for in order to have a 
valid estimation target for V,,(6). Using the fact that 
weighted and unweighted local polynomial regression and 
weighting within cell all produce approximately condi- 
tionally unbiased estimators of the full sample estimator, 
Vu rat = Lies, WYi/Dies, Wj, for the two response pro- 
pensities functions, we decided to use the Jackknife variance 
estimator of Y, , aS a “proxy” for Var(E(6|s,)). Hence, 
our “comparison variance” will be defined as 


9-8) = Vix (Fura) + Fad ow — 6)”. 
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Using ¥ (6) instead of the true variance will tend to 
underestimate any bias issues associated with the use of the 
jackknife variance estimator for the full sample estimator. 
However, it will show how well the replication procedure 
manages to capture the nonresponse variability. 

Table 3 gives relative biases of the Jackknife variance 
estimators obtained in this experiment. The results show that 
the jackknife variance estimator performs reasonably well 
for both nonresponse mechanisms and all estimators 
considered. The weighted local polynomial regression 
adjusted procedure appears to yield estimated variances in 
greater consonance with the comparison variance than when 
the procedure is implemented by its unweighted version. 
The results for the nonlinear predictor function exhibit more 
bias than those for the linear predictor, with more 
pronounced positive and negative biases present for the 
former for all the variables. As discussed in Da Silva and 
Opsomer (2006), replication methods for nonresponse- 
adjusted estimators often ignore a component of the total 
variance, which includes the effect of both sampling and the 
response mechanism. We therefore conjecture that the 
different bias behaviors exhibited for the different variables 
could be due to this missing variance component. 


6. Concluding remarks 


In this article, we studied properties of nonparametric 
propensity weighting as an adjustment procedure for survey 
nonresponse. The local polynomial regression technique is 
seen to offer a flexible way of constructing new survey 
adjustments for nonresponse. The results in the article 
extend those in Da Silva and Opsomer (2006) by allowing 
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the use of local polynomials of arbitrary degree, which 
offers both theoretical and practical advantages over zero- 
degree kernel regression. 

In addition to its good theoretical properties, the 
estimator was shown in the simulation experiment to be 
competitive with an estimator based on a correctly specified 
parametric model in terms of bias and variance, while 
protecting against a potentially misspecified model. The 
weighting-cell estimator is similarly robust against model 
misspecification, but a particular advantage of nonpara- 
metric regression methods over weighting cell approaches is 
the connection to broad classes of modeling techniques 
available in the non-survey literature. Extensions of the 
methodology we described here to semiparametric and 
(generalized) additive models (Hastie and Tibshirani 1986) 
are readily formulated and should work well in a wide range 
of potential response model scenarios, including situations 
with multiple covariates that are both categorical and 
continuous. A detailed discussion of these extensions is 
beyond the scope of the current paper, however. 

In Section 5, we applied the nonparametric nonresponse 
adjustment to NHANES data by modeling the response 
probability as a smooth function of the age of the 
respondents, and weighting the data by the inverses of the 
estimated response probabilities. The same approach can be 
used in other survey datasets whenever continuous 
covariates related to the response probability are available 
for all elements in the original sample. This provides a 
viable alternative to the commonly used weighting-within- 
cell approach for situations in which cells are constructed by 
“binning” one or several continuous variables. 


Relative biases (%) of the Jackknife variance estimators of estimators of the mean systolic blood pressure (SBP), diastolic blood 
pressure (DBP), indicator of hypertension (HTN) and indicator of high serum total cholesterol (HTC), based on 1,000 response sets 
for two propensity functions of the age of the survey participant in NHANES 2005-2006 


Logistic propensity function 


Logistic propensity function 


(linear predictor) (nonlinear predictor) 
Type of adjustment SBP DBP HT HTC SBP DBP HIN HTC 
True Response Propensities 0.55 -0.47 -0.06 0.16 0.92 -0.26 -1.03 -2.76 
Weighted Local Polynomial 
Regression: 
Degree 0 -0.66 233 2.74 4.44 1.63 -2.27 -5.12 -9.44 
Degree | -0.31 -1.03 0.31 1.87 5.27 4.03 2.60 -9.95 
Degree 2 -0.14 -0.76 0.41 0.49 0.25 0.65 -2.60 -3.60 
Degree 3 -0.27 -1.03 0.39 0.48 0.19 0.45 -2.19 -3.02 
Unweighted Local Polynomial 
Regression: 
Degree 0 2.00 Deli B21, 5.56 5.73 0.31 1.83 -10.22 
Degree | DED 1.06 2.63 2.61 7.46 SS) 4.33 -10.43 
Degree 2 2.26 1.07 2.88 1.36 4.16 3.81 1.62 -2.94 
Degree 3 DON 1.01 2.94 1.46 3.45 3.65 0.96 -2.63 
Weighting Within Cell -1.15 1.70 -0.47 5.16 2.69 -6.91 3.06 -5.88 
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There are still a number of open issues that need to be 
further investigated with respect to implementation of the 
method in actual surveys, whether in the univariate case 
described in detail here or in the various model extensions 
just mentioned. An important practical issue is the selection 
of estimator settings such as the degree of the local 
polynomial and the bandwidth. As noted in the non- 
parametric literature (e.g., Fan and Gibels 1996, page 77) 
and also confirmed in the simulations, higher degree 
polynomials reduce the bias but increase the variance, so 
that polynomials of degree k =1 or 2 are generally 
recommended as a good compromise. More critical is the 
choice of bandwidth parameter. In our simulations, the 
results were only modestly sensitive to the choice of 
bandwidth within a “reasonable” range of values, i.e., ones 
ensuring that the number of observations used for estimating 
(x) at any x does not become too small (see discussion at 
the end of Section 2), or that is so large that the fit cannot 
capture changes in $(-) over the range of x. As a rule of 
thumb, we would recommend considering values for h that 
are within 20% and 50% of the range of x as a good place 
to start, and making a final determination by looking at both 
model diagnostics for the model fit (x) and weight 
diagnostics for the adjusted survey weights (x, $,)”, 
similarly as would be done when constructing cell-based 
weights. 
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Appendix 


A.1 Assumptions 


We now State the assumptions needed to derive our main 
results. A detailed discussion of these assumptions is 
provided in Da Silva and Opsomer (2008). Consider the 
asymptotic framework of Section 3. Let J, = (J, 
I,, ..., Ty)’ be the sample inclusion indicator vector for the 
v" population. Suppressing the v for ease of notation, let 
mt, = Prd; = 1), and let 


Bs hee BATT di, -%,, )) (19) 


denote higher moments for the sample inclusion indicators 
I,,J,,...,J, with respect to the sampling design. We 


“fica pi Jk 
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assume that there are positive constants A,,A,,...,A, such 


that: 
(ARP LNG ey oo, Ve Ue 

(A2) N.'n, > nm, forsome 0 < 2 <1, as Vv > &; 
(A3) For distinct 


Hs Jostens I; e Uie where k = 2, 


4 
WAR (N-£+)) nh, if k is even, 
[As Jel 


Pnee (w—e+1)] ni hg if k is odd 


(A4) lim, ,..My' Liev, = HE (0, 0) and Ny" Dieu, 
X,;, forall v 2 1. 


4 
y,|s 


Let R,=(R, R,,....Ry)' denote the response 
indicator vector for the v-th population. In addition to the 
assumptions on the sampling design and the population 
distribution of the variable Y, we will also need the 
following assumptions on the response mechanism: 


(B1) R,, R,,..., Ry, are independent random variables; 


(B2) Prk, = | Ts Vw xa} = 
Prik, =1|x,}=0,VieU,; 


(B3) 6, = o(x,), Vie U,, where 0(-) is a (k +2)" 
continuously differentiable function with 
A, < 0() <1. The first derivative $’(-) has a finite 
number of sign changes. 


Regarding the distribution of the x, and the kernel 
estimator, we assume that: 


(Cl)For all v21, x, x),...,xy are realizations of 
random variables X,, X,,....X, independent and 
identically distributed with distribution Fy (x)= 
{ty dt, where f,(-) is a continuous and 
positive probability density function on a compact 
setlayabel: 

(C2)The kernel function XK(-) is a bounded and 
continuous probability density, which is symmetric 
around zero and supported on [-1, 1]; 


(C3) Clak K@)@au< «: 

(C4)For all v > 1, {h,} is a sequence of bandwidths 
satisfying O<h, <1,h,>0,nk >ow and 
Nyh,/log N, > ©, as v > ~; 

(C5)The first derivative f;(-) is continuously 
differentiable and contains a finite number of sign 


changes on supp(/; ). The first derivative K'(-) has 
a finite number of sign changes on supp(K); 


Survey Methodology, December 2009 


(C6)The matrix N Ty’ 
and all v > 1. 


is non-singular for all i € U, 


A.2 Technical derivations 


Complete proofs are in Da Silva and Opsomer (2008). 
The proof of Theorem | relies on bounding the moments of 
the difference Y,,,.— Y,,g, under the combined design and 
response model probability mechanism, followed by 
deriving the rates of convergence for the bias and variance 
of the linearized estimator y,,,. This is done in a series of 
six lemmas, which are stated here without proof. The proof 
of Theorem 2 is based on the result of Theorem 1, followed 
by an additional linearization of the ratio form. 

For notational simplicity in what follows, we suppressed 
the fact that the results are conditional on the sequences 
X, = (%,-..,Xy,) in the populations U,. However, the 
results in these lemmas are shown to hold with probability 
one over these sequences in Da Silva and Opsomer (2008), 
as was also done in Da Silva and Opsomer (2006). Hence, 
the results can be interpreted to hold for all population 
sequences, except on a set of probability 0 with respect to 
the distribution of the x,. 


Lemma 1. Assume that assumptions (C1)-(C5) hold. 
Consider \1,(K, x) = ,, z'K(z)dz, where D4 = 
V (x + ht)e supp(f, )} A supp(K). Then, for all £ = 0, 
. k +2, 
sup Ke 7%) ~ EQ, £)) > 0, 
xesupp( fy) N. i yy [=n “Jee v>o© 
where 


E(x, £) = fy (x) Hb (K, x) hi 
+ fy (x) Hos) (K, x) A's o( hi"). 


Lemma 2. Assume that assumptions (C1)-(C5) hold. 
Consider the population fit 6, = (x,, ka et) izeeU 
defined in (10). Hence, for all i € U,, there exists positive 
bounded terms c,(x;), ¢,(x;) and c,(x,), such that if x, in 
an interior point of supp( fy ) 


c,(x, Ak? + o(hi**) kis even 
c,(x, ni! + o(hi*"!) kis odd 
and if x, ina boundary point of supp (fy ) 
b, — 6(x,) = ¢5(x,) Ay + o(hy”"), 
where all the smaller order terms hold uniformly in i € U,,. 


Lemma 3. Assume that assumptions (C1) and (C4) hold. 
Then, 
i) For p €[0,) fixed, 
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lim sn 


vo 


P 
» Haensnen < 0, 


Vir cy: jeU, 


uniformly in x; 


11) limsup 


vo 


Ds Tx, €[0,h, }UU-h, 1} < 


vi'y jeu, 


111) imsup — = J prethel-nyy 


vo Vv je 


iv) there exists v', independent of x, such that 


whenever v > Vv’, 


Do Ljenchy 2 +1 
jeU. 


JE v 


Lemma 4. Suppose the assumptions of Theorem 1 hold. 
Consider the matrices Ds = {T,; pq} and T, = {T,, 5} and 
the vectors t,, = it By = {t,,} and B, ={B,,} 
given in (7) a (10). Then, 


i) the N°! T, pq and N, t, » are uniformly bounded 
ing 'U.. forall spe gi= Uys, Kt); 
ii) the T,, ,, and t,, , satisfy 
sg 8 
om MSE 
max E Tisi pq” *ipa | _ O . and 
l<p, qSk+1 N, n hi 


A 8 

Levinas ] 
max md 3 of ats} 
I<psk+l N, Halts 


uniformly in i € U\; 


iii) the random variable e| T,'(t,, — T,, B,) satisfies 


A 1 
maxE (¢| T;'@, - 1,B,)/,8,) = of (20) 
ieU, nh, 


and 


, ] 
max E (e; T,'(t,, — T,, B,)) ‘= of =z] fo) 


Lemma 5. Suppose the assumptions of Theorem 1 hold. 
Then, for all v = 1 
i) the reciprocal of 6, 
ieU,; 


is uniformly bounded in 

ii) the partial derivatives of o;" of orders one up to 
four, when evaluated at T,, = T,, t,, = t,, 8, = 0 
and 5, = 0, are uniformly naghege Sth inieU; 

iii) E(o;*) is uniformly bounded in i € U,; 

iv) the reciprocal of b, Satisfies 
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(22) 


uniformly in i € U,, where the €,, are random 
variables such that 


Lemma 6. Suppose the assumptions of Theorem 1 hold. 
Define the random variables Y 3,5 d ry QF Endy AS 


Pz), aeeys hiv). = 


— >, STAG TB), en) R, 
Then, 
O(hi*©)) _ k even, 
Emr Di a (23) 
Oh") kod, 
Mi l 
Var (Y gy) = of) ; (24) 
ny, 
aa l 
E[d. ], E[d2, A])' =O 25 
(E[d.4,], Elda A) (—] (25) 
and 
1 
E(e*,,) =O 26 
=o Fg a: 
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WAY 


Estimation of the monthly unemployment rate through 
structural time series modelling in a rotating panel design 


Jan van den Brakel and Sabine Krieg ' 


Abstract 


In this paper a multivariate structural time series model is described that accounts for the panel design of the Dutch Labour 
Force Survey and is applied to estimate monthly unemployment rates. Compared to the generalized regression estimator, 
this approach results in a substantial increase of the accuracy due to a reduction of the standard error and the explicit 


modelling of the bias between the subsequent waves. 


Key Words: Small area estimation; Rotation group bias; Survey errors. 


1. Introduction 


The Dutch Labour Force Survey (LFS) is based on a 
rotating panel design. Each month a sample of addresses is 
drawn and data are collected by means of computer assisted 
personal interviewing of the residing households. The 
sampled households are re-interviewed by telephone four 
times at quarterly intervals. The estimation procedure of this 
survey is based on the generalized regression (GREG) 
estimator, developed by Sarndal, Swensson and Wretman 
(1992). 

Due to the following properties, GREG estimators are 
very attractive to produce official releases in a regular 
production environment and are therefore widely applied by 
national statistical institutes. First, GREG estimators are 
approximately design-unbiased, which provides a form of 
robustness in the case of large sample sizes. These 
estimators are derived from a linear regression model that 
specifies the relationship between the values of a certain 
target parameter and a set of auxiliary variables for which 
the totals in the finite target population are known. If this 
linear regression model explains the variation of the target 
variable reasonably well, then this might reduce the design 
variance as well as the bias due to selective nonresponse, 
Samdal and Swensson (1987), Bethlehem (1988), and 
Samdal and Lundstrém (2005). Model misspecification, on 
the other hand, might result in an increase of the design 
variance but the point estimates remain approximately 
design unbiased. Second, GREG estimators are often used 
to produce one set of weights for the estimation of all target 
parameters of a multi-purpose sample survey. This is not 
only convenient but also enforces consistency between the 
marginal totals of different publication tables. 

There are two major problems with the rotating panel 
design of the LFS and the way that the GREG estimator is 
applied in the estimation procedure. First, there are 


substantial systematic differences between the subsequent 
waves of the panel due to mode- and panel effects. This is a 
well-known problem for rotating panel designs, and is in the 
literature referred to as rotation group bias (RGB), see 
Bailar (1975). In the LFS, the level of the unemployment 
rate in the subsequent waves is substantially smaller 
compared to the first wave. There are also systematic 
differences between the seasonal effects of the subsequent 
waves. 

A second problem is that the monthly sample size of the 
LFS is too small to rely on the GREG estimator to produce 
official statistics about the monthly employment and 
unemployment. GREG estimators have a relatively large 
design variance in the case of small sample sizes. Therefore, 
in the LFS, each month the samples observed in the 
preceding three months are used to estimate quarterly 
figures about the labour market situation. The major 
drawback of this approach is that the real monthly seasonal 
pattern in the unemployment rate is smoothed out. Also 
structural changes in unemployment appear delayed in the 
series of quarterly figures. 

Since the monthly sample size is too small to apply 
design-based or direct survey estimators, model-based esti- 
mation procedures might be used to produce sufficiently 
reliable statistics. In the case of continuously conducted 
surveys, a structural time series model can be applied to use 
information from preceding samples to improve the 
accuracy of the estimates. This model can be extended to 
account for the RGB and the autocorrelation (AC) between 
the different panels of the LFS. This approach makes 
efficient use of the rotating panel design of the LFS in 
estimating monthly figures about the labour market, 
and is originally proposed by Pfeffermann (1991) and 
Pfeffermann, Feder and Signorelli (1998). These techniques 
are applied in this paper to estimate the monthly unemploy- 
ment rate of the LFS. Other references to authors that apply 
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time series models to develop estimates for periodic surveys 
are Scott and Smith (1974), Scott, Smith and Jones (1977), 
Tam (1987), Binder and Dick (1989, 1990), Bell and 
Hillmer (1990), Tiller (1992), Rao and Yu (1994), 
Pfeffermann and Burck (1990), Pfeffermann and Rubin- 
Bleuer (1993), Pfeffermann and Tiller (2006), Harvey and 
Chung (2000), and Feder (2001). 

Composite estimators can be considered as an alternative 
to time series models. They are developed under the 
traditional design-based approach, to use information 
observed in previous periods from periodic surveys with a 
rotating panel design, to improve the precision of level and 
change estimates. Some key references to composite esti- 
mators are Hansen, Hurwitz and Meadow (1953), Rao and 
Graham (1964), Gurmmey and Daly (1965), Cantwell (1990), 
Singh (1996), Gambino, Kennedy and Singh (2001), Singh, 
Kennedy and Wu (2001) and Fuller and Rao (2001). 

In Section 2, the survey design of the LFS is summarised. 
A structural time series model that accounts for the rotating 
panel design of the LFS is developed in Sections 3 and 4. 
The results are detailed in Section 5. Some general remarks 
are made in Section 6. 


2. The dutch Labour Force Survey 


2.1 Sample design 


The objective of the Dutch LFS is to provide reliable 
information about the labour market. Each month a sample 
of addresses is selected from which households are 
identified that can be regarded as the ultimate sampling 
units. The target population of the LFS consists of the non- 
institutionalised population aged 15 years and over residing 
in the Netherlands. The sampling frame is a list of all known 
occupied addresses in the Netherlands, which is derived 
from the municipal basic registration of population data. 
The LFS is based on a stratified two-stage cluster design of 
addresses. Strata are formed by geographical regions. 
Municipalities are considered as primary sampling units and 
addresses as secondary sampling units. All households 
residing at an address, up to a maximum of three, are 
included in the sample (in the Netherlands, there is generally 
one household per address). Since most target parameters of 
the LFS concern people aged 15 through 64 years, addresses 
with only persons aged 65 years and over are undersampled. 

In October 1999, the LFS changed from a continuous 
survey to a rotating panel design. In the first wave, data are 
collected by means of computer assisted personal inter- 
viewing (CAPI). For all members of the selected house- 
holds, demographic variables are observed. For the target 
variables only persons aged 15 years and over are inter- 
viewed. When a household member cannot be contacted, 
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proxy interviewing is allowed by members of the same 
household. Households, in which one or more of the 
selected persons do not respond for themselves or in a proxy 
interview, are treated as nonresponding households. The 
respondents aged 15 through 64 years are re-interviewed 
four times at quarterly intervals by means of computer 
assisted telephone interviewing (CATI). During these re- 
interviews a condensed questionnaire is applied to establish 
changes in the labour market position of the respondents. 
Proxy interviewing is also allowed during these re- 
interviews. The monthly gross sample size averaged about 
8,000 addresses when the LFS first changed to a rotating 
panel design. The monthly sample size gradually declined to 
about 6,500 addresses in 2008. During this period about 
65% completely responding households are obtained. 


2.2 Rotation group bias 


The rotating panel design, described in Section 2.1, results 
in systematic differences between the estimates of the 
unemployment rate of the successive waves in one time 
period. In the literature, this phenomenon is known as RGB, 
see e.g., Bailar (1975), Kumar and Lee (1983) and 
Pfeffermann (1991). The RGB in the LFS results in a 
systematic underestimation of the level of the unemployment 
rate in the CATI waves but also in systematic differences 
between the seasonal patterns. The RGB is a consequence of 
the following strongly confounded factors: 


— Selective nonresponse between the subsequent waves, 
i.e., panel attrition. 

— Systematic differences between the populations that are 
reached with the CAPI and CATI modes. It is anticipated 
that these differences are relatively small, since 
telephone numbers are asked during the first interview. 
As a result, secret numbers and cell-phone numbers are 
also called. 

— Mode-effects, i.e., systematic differences in the data due 
to the fact that the interviews are conducted by telephone 
instead of face to face. Under the CAPI mode the 
interview speed is lower, respondents are more engaged 
with the interview and are more likely to exert the 
required cognitive effort to answer questions carefully. 
Also less socially desirable answers are obtained under 
the CAPI mode due to the personal contact with the 
interviewer. As a result, less measurement errors are 
expected under the CAPI mode (Holbrook, Green and 
Krosnick 2003, and Roberts 2007). Van den Brakel 
(2008) describes an experiment where the CAPI and 
CATI data collection modes are compared in the first 
wave of the LFS. It follows that the estimated 
unemployment rate is significantly smaller under the 
CATI mode. 


Survey Methodology, December 2009 


— The fraction of proxy interviews is larger under the 
CATI mode (Van den Brakel 2008). This might result in 
an increased amount of measurement errors. 

— Effects due to differences between the CAPI question- 
naire and the CATI questionnaire. The CATI 
questionnaire is a strongly condensed version of the 
CAPI questionnaire since the re-interviews focus on 
changes in the labour market position of the respondents. 

— Panel effects, i.e., systematic changes in the behaviour of 
the respondents in the panel. For example, questions 
about activities to find a job in the first wave might 
increase the search activities of the unemployed 
respondents in the panel. Respondents might also adjust 
their answers in the subsequent waves systematically, 
since they learn how to keep the routing through the 
questionnaire as short as possible. 


It is assumed that the estimates based on the first wave 
are the most reliable, since CAPI generally results in a 
higher data quality and the first wave does not suffer from 
the panel effects mentioned above. In order to minimize the 
effects of the RGB, the second, third, fourth and fifth waves 
are currently calibrated to the first wave as will be described 
in Section 2.3. 


2.3 Regular estimation procedure 


Target parameters about the employment and un- 
employment are defined as population totals or as ratios of 
two population totals. The unemployment rate, which is 
investigated in this paper, is defined as the ratio of the total 
unemployment to the total labour force. This population 
parameter is estimated as the ratio of the GREG estimate for 
the total unemployed labour force to the estimated total 
labour force. Each month estimates about the employment 
and unemployment for the preceding three months are 
published. 

In an attempt to correct for the RGB, a rather laborious 
weighting procedure is used in the regular estimation 
procedure. The most important steps are summarized here. 
First, the inclusion probabilities are derived, which reflect 
the sampling design described above as well as the different 
response rates between geographical regions. Subsequently, 
the inclusion weights of each CATI wave are calibrated 
with the GREG estimator to the labour force status observed 
in the first wave. In the next step, the calibrated weights of 
the CATI waves and the inclusion weights of the CAPI 
wave are used as the design or starting weights of the 
GREG estimator, using a weighting scheme that is based on 
a combination of different socio-demographic classifica- 
tions. The integrated method for weighting persons and 
families of Lemaitre and Dufour (1987) is applied to obtain 
equal weights for persons belonging to the same household. 
Finally, a bounding algorithm proposed by Huang and 
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Fuller (1978) is applied to avoid negative weights. This 
estimation procedure is conducted with the software 
package Bascula, Nieuwenbroek and Boonstra (2002). 

Since this weighting procedure hardly corrects for the 
RGB, an additional rigid correction is applied. For the most 
important parameters the ratio between the estimates based 
on CAPI only and the estimates based on all waves is 
computed using the data of 12 preceding quarters. Estimates 
for the preceding three months are multiplied by this ratio to 
correct for RGB. 


2.4 Monthly GREG estimates based on monthly 
data 


In Section 3, a structural time series model is developed 
to estimate the monthly unemployment rate. The input data 
for this time series model are the GREG estimates for the 
monthly unemployment rate using the monthly sample data 
of the separate waves. Let 8, denote the true but unknown 
unemployment rate for month ¢. Now Y,” denotes the 
GREG estimate of the unemployment rate of month f, 
based on the sample which entered the panel in month 
t — j. For the period of January 2001 until December 2008 
each month five independent GREG estimates for the same 
parameter 98, are produced, using the five separate waves 
that are observed each month, ie., Wey. for aje= 0733699; 
12. These estimates are defined as 
es 
yuo, (2.1) 


a | 
t, St! 


with ¢,/ and ¢,/ the GREG estimates for the unemployed 
labour farce rel the labour force at time /¢, based on the 
sample that entered the panel at ¢ — /. 

The separate monthly waves are weighted with a reduced 
version of the weighting scheme that is applied in the 
regular weighting procedure for the quarterly figures. The 
estimates based on the CATI data are not adjusted to correct 
for RGB, since a multivariate time series model is applied to 
correct for this bias. 

The variance of (2.1) can be estimated with 


vary) ))= 


H i Nn, n * 
ae At 
JP otal 2a | Sone side = [Sn (2.2) 
h= 11, 1 ii, 


with 


Ve” Custer: Suz) 


mM 
{- t-j T 
/ = Dain: oi x,,b,,) — 
[=I 


Here yi)! is a binary variable taking value one if the /" 
person belonging to the k™ household that entered the 
sample at time ¢— j belongs to the unemployed labour 
force at time ¢ and zero otherwise, z;,,/ a binary variable 


taking value one if the / person of the k" household 
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belongs to the labour force at time ¢ and zero otherwise, 
x,, a vector with the auxiliary information of the / "person 
belonging to the k" household used in the weighting 
scheme of the GREG estimator, b,, and b, the regression 
coefficient of the regression function of y,/ respectively 
zy, ON X,, Ww, the regression weight of household k, 
n,; the number of completely responding households of 
stratum h = 1, ...,H, at time ¢ of the sample that entered 
the panel at ¢ — j, and m, the number of persons aged 15 
years and over belonging to the k™ household. Recall from 
Section 2.3 that persons belonging to the same household 
have equal weights due to the application of the integrated 
method for weighting persons and families of Lemaitre and 
Dufour (1987). Formula (2.2) is the variance estimation 
procedure implemented in Bascula to approximate the 
variance of the ratio of two GREG estimators. 

The estimates for the monthly unemployment rate 
obtained with the structural time series approach will be 
compared in Section 5.3 with monthly estimates based on 
the GREG estimator using the data observed in the five 
waves. For this comparison a slightly simplified version of 
the procedure described in Section 2.3 is applied to combine 
the data observed in the different waves to obtain monthly 
GREG estimates. First, a GREG estimate Y, is computed 
using the data observed in the five waves using the same 
weighting procedure used in the regular production process 
to estimate quarterly figures, see Section 2.3. The weighting 
scheme is slightly simplified because less data are available. 
Subsequently a correction factor based on the preceding 
three years is computed as: 


ee ees (2.3) 


Finally, the corrected estimate is computed: 


a We 


t 


(2.4) 


Because the series start at January 2001, c, can be 
computed from December 2003. To get a corrected GREG 
estimate for all months, Cp.-ember2003 1S uSed in formula (2.4) 
for the periods preceding December 2003. The variance of 
(2.4) is approximated by var(Y°) = c? var(¥,), where 
var(Y,), is computed with formula (2.2), using the data of 
all waves accordingly. 


3. Time series model 


Direct estimators, like the Horvitz-Thompson estimator 
or the GREG estimator, assume that the monthly 
unemployment rate 8, is a fixed but unknown population 
parameter. Under this design-based approach, an estimator 
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for 9, for cross-sectional surveys only uses the data 
observed at time ¢. Data from the past are only used in the 
case of partially overlapping samples in a panel design, but 
not in the case of repeatedly conducted cross-sectional 
designs. Scott and Smith (1974) proposed to consider the 
population parameter 6, as a realization of a stochastic 
process that can be described with a time series model. 
Under this assumption, data observed in preceding periods 
t—1,t-—2, ..., can be used to improve the estimator for 
8, even in the case of non-overlapping sample surveys. 
Recall from Section 2.4 that ¥'/ denotes the GREG 
estimator for 6, based on the panel observed at time f, 
which entered the survey for the first time at ¢ — 7. Due to 
the applied rotation pattern, each month a vector Y, = 
(YO Vii yaya? )" is observed s According, to 
Pfeffermann (1991), this vector can be modelled as 


Y 41.0) eye? (341) 
with 1, a five dimensional vector with each element equal 
tofioney A= Cane? AAO and slyo= (yo yi, brane Je 
vectors with time dependent components that account for 
the RGB in the trend and the RGB in the seasonal 
components respectively, and e,=(e ef ef ® el? ef)! 
the corresponding survey errors for each panel estimate. 
Time series models for the different components in (3.1), 
i.e., the population parameter 8, the RGB for the trend 2, 
the RGB for the seasonal patterns y,, and the survey errors 


e,, are developed in Sections 3.1 through 3.3. 


3.1 Time series model for the population parameter 


With a structural time series model, the population 
parameter 8, in (3.1) can be decomposed in a trend 
component, a seasonal component, and an_ irregular 
component, i.e.: 


Osu aS ick s 3 (3:2) 
where L, denotes a stochastic trend component, S, a 
stochastic seasonal component, and ¢, the wregular 
component. For the stochastic trend component the so- 
called local linear trend model is used, which is defined by 
the following set of equations: 


L, = Li a RK, ae Lae 
R, 7 RS + Nr 
Ga Natta? 
E(n,,1) = 0, Cov(n, ;; N21) - 
Owieett sf) 
Op eiht= ft 
E(nr 1) a 0, Cov(ne ps Nei) - GB) 
Ova Hor Aig: 
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The parameters L, and R, are referred to as the trend and 
the slope parameter respectively. The seasonal component is 
modelled with the trigonometric form 


(3.4) 
(fe | 
where 
Si, = 5,4 cos(h,) + ‘oe sin(h,) + @, , 
Si, = S,,.cos(h,) — S,,, sin(h,) 
a Giese es 
h, = a l E iF 70; 
6 
E(o,,) = E(@,,) = 0, 
Cov(@, ,,@p ) = Cov(, by /, ) 
Gee kifgle=nieandiity—s 
O abaciihe ly Halak Otel tcl 
Cov(@,,,@,,) = 0 for all / and 1. (3.5) 


The irregular component ¢, contains the unexplained 
variation and is modelled as a white noise process: 
Gait 1, 

E(é,) = 0, Cov(é,, &)) = 
Ow it rs fr 


(3.6) 


3.2 Time series model for rotation group bias 


The systematic differences between the trend and the 
seasonal components of the subsequent waves are modelled 
in (3.1) with A, and y,. Additional restrictions for the 
elements of both vectors are required to identify model 
(3.1). Here it is assumed that an unbiased estimate for 8, is 
obtained with the first wave, which is observed by CAPI, 
i.e., Y/. This implies that the first component of 4, and y, 
equals zero. Now i, measures the systematic differences in 
the trend of the second, third, fourth and fifth wave with 
respect to the first wave. The components of 4, are defined 
as: 


Pea meh Veter) 550.9212, a1 a.) 


E(m, 4.) = 9, 
2 


go, Mubr=wrandyi=>/' 
Cov(ny, 51 Nha, 7’, = 
0 ifft#torj # j’ 


Furthermore y, measures the systematic differences in 
the seasonal components with respect to the first wave. This 
implies that y? = 0. The other components of y, are 
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defined as trigonometric functions, which are of the form of 
(3.5). The variance of the disturbances of the seasonal 
components are assumed to be equal for all waves and is 
denoted by oy. 

To borrow information across the panel waves, the RGB 
for the trend as well as the RGB for the seasonal 
components are modelled as time invariant components, i.e., 
oy om = 0. As a kind of model diagnostic, the model 
initially allows for time dependent components. The 
maximum likelihood estimates for o; and os are close to 
zero in this application. If this is not the case, it might be 
possible to allow for separate time independent RGB 
components for different time intervals. 


3.3. Time series model for the survey errors 


Finally a time series model for the survey errors in (3.1) 
is developed, which uses the direct estimates for the 
variance and AC’s for the survey errors of the different 
panels as prior information. From (3.1) it follows that the 
survey errors for the first wave are defined as e/ = Y' — 0. 
For the second, third, fourth and fifth wave, they are defined 
dstemSuy OS Wd yd» for gc 3.659 :12: 

Direct estimates for the variances of the survey errors for 
the separate panels are obtained with (2.2). These estimates 
are smoothed by modelling the variance estimates for the 
separate panels with a linear regression model Var(Y,/) = 
bi +b) (Y/7/n'-’) + error, where n// denotes the sample 
size at time ¢ of the sample that entered the panel at ¢ — /. 

The rotating panel design implies sample overlap with 
panels observed in the past. The sample of the first wave 
enters the panel for the first time at time 4 so there is no 
sample overlap with panels observed in the past. 
Consequently, the survey errors of the first wave, e, are 
not correlated with survey errors in the past. The survey 
error of the second wave, i.e, | is correlated with the 
survey error of the first wave that entered the panel three 
months earlier, i.e., Cs . In a similar way, the survey error 
of the third wave, ie., e/ °, is correlated with e/} and 
e/°. The survey error of the fourth wave, ie, e’, is 
correlated with e!-}, e!-2 and ef->. Finally, the survey error 
of the fifth wave, ie, ef", is correlated with 
G3 6&9 and e,)>. 

The AC’s between the survey errors of the subsequent 
waves are estimated using the approach proposed by 
Pfeffermann etal. (1998). Since the real survey errors 
cannot be observed directly, this approach starts with 
calculating the autocovariances for the pseudo survey errors, 
which are defined as (Y,°’ —Y,), where Y, denotes the 
average of the five panel estimates Y'/ at time ¢. The 
autocovariances of the pseudo survey errors for a separate 
wave are influenced by the autocovariances of the real 
survey errors of the other waves, since the pseudo survey 


Statistics Canada, Catalogue No. 12-001-X 


182 van den Brakel and Krieg: Estimation of the monthly unemployment rate through structural time series modelling 


errors are defined as the deviation of a panel estimate with 
the average of all panel estimates obtained at time ¢. 
Equation (4) of Pfeffermann etal. (1998) specifies the 
relation between the autocovariances of the pseudo survey 
errors and the real survey errors. From this equation, it 
follows that the autocovariances of the real survey errors 
can be derived from the autocovariances of the pseudo 
survey errors by @, = F 'C,, with C, a vector containing 
the five autocovariances of the pseudo survey errors at lag 
k, @, a vector containing the five autocovariances of the 
survey errors at lag k, and F a MxM _ dimensional 
matrix where the diagonal elements equal (M -1/M)° 
and the off-diagonal elements (1/M)°. Here M denotes 
the number of waves of the panel design (M = 5 in this 
application). The AC’s and the partial autocorrelations 
(PAC) of the survey errors of the subsequent waves are 
given in Table 3.1. 


Table 3.1 
Correlations and partial autocorrelations for the survey errors of 
the separate panels 


wave la 
1 2 3 4 
] AC -0.029 0.264 0.022 0.230 
PAC -0.029 0.263 0.038 0.175 
2 AC 0.291 0.135 0.035 -0.250 
PAC 0.291 0.054 -0.020 -0.287 
3 AC 0.240 0.120 0.087 0.219 
PAC 0.240 0.066 0.047 0.194 
4 AC 0.442 0.253 0.122 0.156 
PAC 0.442 0.072 -0.016 0.115 
5 AC 0.249 0.298 -0.183 0.127 
PAC 0.249 0.252 -0.344 0.218 
Mean* AC 0.306 0.224 -0.030 Ona 
PAC 0.306 0.144 -0.150 0.162 


Underlined AC’s and PAC’s refer to waves with sample overlap 
*): Means are based on the waves with sample overlap. 


The standard errors of the estimated AC’s equal 1/JT, 
where 7 denotes the number of observations. This implies 
that correlations with an absolute value larger than 0.21 are 
significantly different from zero at a 5% significance level. 
The lags in Table 3.1 refer to three months periods, so lag 
one equals a time lag of three months, lag two a time lag of 
six months, efc. 

The AC’s in Table 3.1, which are based on overlapping 
samples, are underlined. The AC’s for the overlapping 
samples are positive as might be expected. An exception is 
the AC at lag three for the fifth wave, which has a negative 
value. This correlation, however, is not significantly 
different from zero. The AC’s for lag one of the overlapping 
samples are all significantly different from zero. For lag 
two, the AC’s of the overlapping samples are significantly 
different from zero for the fourth and the fifth wave, but not 
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for the third wave. The AC’s that are based on non- 
overlapping samples are sometimes unexpectedly large, e.g., 
lag two and four of the first wave and lag four of the third 
wave. The AC for lag four of the second wave, on the other 
hand, has a surprisingly large negative value. 

Pfeffermann et al. (1998) also report large positive AC’s 
for lags with non overlapping samples. In their case this can 
be explained since samples are replaced in small geograph- 
ical regions. In the Dutch LFS sample replacement takes 
place at the national level. There is no good explanation 
why the AC’s for the non overlapping samples are 
sometimes small and sometimes take significant positive as 
well as negative values. To obtain more stable estimates, the 
AC’s are averaged over the waves which are based on 
overlapping samples. Thus the mean AC for lag one is the 
average of the AC for the second, third, fourth and fifth 
wave, etc. The values are reported in the last two rows of 
Table 3.1. The standard errors of the PAC’s of order p + 1 
and higher for an AR(p) equal Liar: Box and Jenkins 
(1970). This implies that the PAC’s are not significantly 
different from zero for lags two and higher if an AR(1) 
model with a correlation coefficient of 0.306 is assumed to 
capture the AC of the survey errors for the second, third, 
fourth and fifth wave. 

The direct estimates for the variance and covariance 
structure of the survey errors are combined in the time series 
model using the following general form of the survey 
error model e// = kj /é/ where k/-/ = ,/Var(Y,"), see 
Binder and Dick (1990). This allows for non homogeneous 
variance in the survey errors, that arise e.g., due to the 
gradually decreasing sample size over the last decade. 

Since the first wave is uncorrelated with survey errors 
obtained in the past, it is assumed that & is white noise with 
E(é') = 0 and Var(é)) = 1. As a result, the variance of 
the survey error equals Var(e!) = (k/)’, which is equal to 
the direct estimate of the variance of the GREG estimate for 
the first wave. For the second, third, fourth and fifth wave, it 


is assumed that é"’ = pé_/ + vi /, with p = 0.306, and 


. ren nary 
EE (Vea Us COW WW a av cae 
Wma say fm 
Since &/’ is an AR(l) process, Var(é/) = 
o./(1—p’). To enforce that Var(e'’) equals the direct 
estimate for the variance of the GREG estimate, it follows 
that o. = (1- p’). 


3.4 Final time series model for the monthly 
unemployment rate 


The time series model for the vector with GREG esti- 
mates Y, is obtained by inserting the different components 
developed in Sections 3.1 through 3.3 into (3.1). This model 
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uses the five monthly GREG estimates as input data to 
obtain model-based estimates for the monthly unemploy- 
ment rate. The component for the population parameter 0, 

in (3.2), developed in Section 3.1, takes advantages of 
sample information observed in the past to improve the 
precision of the estimated monthly unemployment rate. The 
components for the RGB, developed in Section 3.2, account 
for the systematic differences between the five monthly 
GREG estimates to avoid that the estimated monthly un- 
employment rate is incurred with this bias. The component 
for the survey errors, developed in Section 3.3, accounts for 
the AC between the five GREG estimates that are based on 
the same sample, observed with quarterly intervals. 
Although this approach is model-based, it accounts for the 
complexity of the survey design of the LFS, since the 
GREG estimates are used as input data. 


4. State space representation 


The time series model for the five monthly GREG 
estimates developed in Section 3 can be expressed in the 
state space representation, see Harvey (1989) or Durbin and 
Koopman (2001). A state space model consists of a 
measurement equation and a transition equation. The 
measurement equation, which is sometimes also called the 
signal equation, specifies how the observations depend on a 
linear combination of the state vector that contains the 
unobserved state variables for the trend, seasonal, RGB and 
the survey errors. The transition equation, which is 
sometimes also referred to as the system equation, specifies 
how the state vector evolves in time. The state space 
representation of the model developed in Section 3 is given 
by Van den Brakel and Krieg (2009). 

Under the assumption of normally distributed error 
terms, the Kalman filter can be applied to obtain optimal 
estimates for the state vector. Estimates for state variables 
for period ¢ based on the information available up to and 
including period ¢ are referred to as the filtered estimates. 
The filtered estimates of past state vectors can be updated, if 
new data become available. This procedure is referred to as 
smoothing and results in smoothed estimates that are based 
on the completely observed time series. So the smoothed 
estimate for the state vector for period ¢ also accounts for the 
information made available after time period ¢. In this paper, 
the Kalman filter estimates for the state variables are 
smoothed with the fixed interval smoother. See Harvey 
(1989), and Durbin and Koopman (2001) for technical 
details. 

The analysis is conducted with software developed in Ox 
in combination with the subroutines of SsfPack 3.0, see 
Doomik (1998) and Koopman, Shephard and Doomik 
(2008). All state variables are non-stationary with the 
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exception of the survey errors. The non-stationary variables 
are initialised with a diffuse prior, i.e., the expectation of the 
initial states are equal to zero and the initial covariance 
matrix of the states is diagonal with large diagonal elements. 
The survey errors are stationary and therefore initialised 
with a proper prior. The initial values for the survey errors 
are equal to zero and the covariance matrix is available from 
the model developed for the survey errors in Section 3.3. In 
Ssfpack 3.0 an exact diffuse log-likelihood function is 
obtained with the procedure proposed by Koopman (1997). 


5. Results 


5.1 Preliminary analyses 


With the GREG estimator monthly estimates for the 
unemployment rate are obtained for each wave as described 
in Section 2.4. In Figure 5.1 the unemployment rate based 
on the CAPI wave is compared with the average of the four 
CATI waves. The graph shows that the unemployment rate 
observed with the first wave is systematically higher than 
for the other four waves. 

The five time series obtained with the different waves are 
modelled with the time series model proposed in Sections 3 
and 4. Preliminary analyses indicate that the estimates for 
the RGB of the seasonal effects in the second wave are not 
significantly different from zero and the RGB for the 
seasonal effects of the third, fourth and fifth wave are not 
significantly different from each other. Therefore the model 
is simplified to one with a single RGB seasonal effect. See 
Van den Brakel and Krieg (2009) for the state space 
representation. 


» 
Ss 
ro 
iS 
re 
& 
o 
= 
> 
& 
a 
= 
cs) 
i=] 
=) 


2001 2002 2003 2004 2005 2006 2007 
Time 


Capi wave — — Mean cati waves 


Figure 5.1 RGB monthly unemployment rate based on GREG 
estimates 


5.2 Estimation results for the time series model 


Maximum likelihood estimates for the hyperpara- 
meters, i.e., the variance components of the stochastic 
processes for the state variables are obtained using a 
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numerical optimization procedure (BFGS algorithm, 
Doornik 1998). To avoid negative variance estimates, the 
log-transformed variances are estimated. The maximum 
likelihood estimates for the log-transformed variance of 
the level of the trend (07), the seasonal component 
(o2), the RGB of the trend (o;) and the RGB of the 
seasonals tend (co; ) tend to large negative values with 
extremely large standard errors. These variance compo- 
nents are therefore put to zero in the final model. The 
estimation results for the remaining hyperparameters are 
presented in Table 5.1. 


Table 5.1 
Maximum likelihood estimates hyperparameters 


Hyperparameter Ln-transformed 


variance comp. 
Estimate St. error Estimate 95% conf. interval 


Lower b. Upper b. 
Slope (o;) -17.226 0.549 0.182E-3 0.106E-3 0.311E-3 
Irregular comp. (co?) -13.480 0.482 1.183E-3 0.737E-3 1.897E-3 


Variance components 


The smoothed Kalman filter estimates for the un- 
employment rate 9, are given in Figure 5.2. These are the 
estimates for the monthly unemployment rate, based on the 
smooth trend model and a seasonal component, corrected 
for the RGB between the five GREG estimates. The local 
linear trend model simplified to a smooth trend model since 
o; = 0. The trend component is time dependent since the 
maximum likelihood estimate of the hyperparameter for the 
slope is positive (see Table 5.1). The seasonal component is 
also time independent, since o7 = 0. Therefore the esti- 
mated seasonal effects obtained with the trigonometric form 
are exactly the same as the results obtained with the well 
known dummy variable seasonal model. The smoothed 
Kalman filter estimates for the trend and the seasonal 
component are plotted in Figures 5.3 and 5.4 respectively. 
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Figure 5.2 Smoothed Kalman filter estimates for the monthly 
unemployment rate 
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Figure 5.3 Smoothed Kalman filter estimates for the trend of the 
monthly unemployment rate 
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Figure 5.4 Smoothed Kalman filter estimates for the seasonal 
effect of the monthly unemployment rate 


The Kalman filter estimates for the RGB of the trend are 
time independent. The smoothed Kalman filter estimates for 
the RGB are given in Table 5.2. The model beautifully 
detects a slightly increasing bias in the trend of the 
subsequent waves. The estimates for the RGB of the four 
CATI waves are significantly different from zero. 


Table 5.2 
Smoothed Kalman filter estimates RGB trend 
Wave RGB St. error 
2 -0.75 0.04 
5: -0.86 0.04 
4 -0.96 0.05 
5 -1.10 0.05 


An interesting empirical result of this application is the 
finding of the seasonality in the RGB. The Kalman filter 
estimates for the RGB of the seasonal effects are also time 
independent. Therefore, a sequence of likelihood ratio tests 
is conducted to reach the finally selected model and to test 
whether the seasonality effects in the RGB of this model are 
jointly significantly different form zero. Consider the 
following nested models: 
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— M1: separate and fixed RGB in the seasonality for 
wave two, three, four and five 

— M2: equal to M1 where the RGB in the seasonality 
of wave two is equal to zero 

— M3: equal to M2 with equal RGB in the seasonality 
of wave three, four and five 

— M4: RGB in the seasonality of wave two, three, four 
and five is equal zero 


The results of the likelihood ratio tests of this sequence of 
models are specified in Table 5.3. 


Table 5.3 
Likelihood-ratio tests for RGB in seasonality 
Model Log Null Likh. ratio _—D-f. p-value 


likelihood hypothesis stat. 
M1 1,592.9 


M2 1,585.5 M2=MI1 14.7 11 0.19568 
M3 SyU3al/ M3 = M2 234 22 0.36422 
M4 15929) M4 = M3 27.6 1] 0.00373 


Testing the hypothesis that M2 equals M1 shows that the 
seasonality of the second wave is not significantly different 
from the first wave. Testing the hypothesis that M3 equals 
M2 shows that the RGB in the seasonality of the third, 
fourth and fifth wave are not significantly different. Testing 
the hypothesis that M4 equals M3 shows that the RGB of 
seasonal effects in last three waves are jointly significantly 
different from zero. 

The smoothed Kalman filter estimates for the RGB of the 
seasonal effects for wave three, four and five are given in 
Figure 5.5. The smoothed Kalman filter estimates of the 
seasonal effects are compared with the smoothed estimates 
for the RGB of the seasonal effects in Figure 5.6. 
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Figure 5.5 Smoothed Kalman filter estimates for the RGB of the 
seasonal effects in the third, fourth and fifth wave 
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Figure 5.6 Comparison of smoothed Kalman filter estimates for 
the RGB of the seasonal effects in the third, fourth and 
fifth wave and the seasonal effects in 2008 


It follows from Figure 5.5 that the seasonal effects in 
February, March, April, August and October in the third, 
fourth and fifth wave are significantly different from the 
first and the second wave. Figure 5.6 shows that the RGB in 
the seasonal effects largely nullifies the seasonal effects in 
these months. The seasonal effects in the last three waves 
are, apparently, less pronounced than in the first two waves. 
The different factors that contribute to the RGB in both 
the trend and the seasonal patterns are summarised in 
Section 2.2. 


5.3. Comparison with GREG estimates 


In this section the monthly GREG estimates for the 
unemployment rate and their standard errors are compared 
with the filtered model estimates. The filtered estimates are 
used since they are based on the complete set of information 
that would be available in the regular production process to 
produce a model-based estimate for the monthly unemploy- 
ment rate for month ¢. 

The GREG estimates based on the CAPI wave for the 
monthly unemployment rates are compared with the filtered 
model estimates in Figure 5.7. Some of the peaks and dips 
in the series of the GREG estimates are partially considered 
as survey errors under the structural time series model and 
flattened out in the filtered estimates for the series. Some of 
these peaks and dips are preserved since they are considered 
as seasonal effects under the time series model. It also 
follows that the filtered estimates are corrected for the RGB 
since the filtered series is at the same level as the series of 
the GREG estimates based on the CAPI wave. This is 
enforced with the assumption that the model parameters for 
the RGB for the first wave are zero (Section 3.2). This 
implies that the CATI waves are benchmarked to the 
outcomes of the first wave. 
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Figure 5.7 Filtered estimates and GREG estimates based on the 
CAPI wave for the monthly unemployment rate 


2007 2008 


The procedure applied in the regular estimation 
procedure of the LFS, to combine the CATI and the CAPI 
waves, is also used to estimate monthly unemployment 
figures. The GREG estimates for the monthly unemploy- 
ment rates based on the five waves, using formula (2.4), are 
compared with the filtered estimates in Figure 5.8. Both 
estimates for the monthly unemployment rate follow the 
same level, since they are both benchmarked to the 
outcomes of the first wave. The GREG estimator is 
benchmarked in a rather ngid way using ratio (2.3), which is 
assumed to be constant in advance over a period of three 
years. The filtered estimates are benchmarked in a more 
subtle way through the explicit modelling of the trend and 
the seasonality in the RGB. The seasonality in the RGB 
indicates that the assumption of a constant RGB is not 
tenable. The monthly GREG estimates based on all waves 
are also compared with the GREG estimates based on the 
CAPI wave in Figure 5.9. 

The ratio correction applied in formula (2.4) to the 
GREG estimates based on all waves removes the RGB in 
the trend, but does not correct for the RGB in the seasonal 
patterns. This follows from Figure 5.8 and 5.9. The series of 
the GREG estimates based on all waves follows the same 
level as the GREG estimates based on the CAPI wave 
(Figure 5.9). There are, however, subtle differences between 
the filtered estimates and the GREG estimates based on all 
waves (Figures 5.8). They partially arise because some of 
the dips and peaks in the GREG estimates are considered as 
survey errors by the time series model but they are also the 
result of systematic differences in the seasonal patterns 
between the subsequent waves. For example, the model 
estimates in February and March are larger in 2003, 2005 
and 2006, and smaller in August in 2004, 2005 and 2006. 

The standard errors of the monthly GREG estimates 
based on all waves, the CAPI wave and the filtered 
estimates are compared with each other in Figure 5.10. The 
standard errors for the GREG estimates are computed as 
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described in Section 2.4. Standard errors of the filtered 
estimates are obtained by the standard recursion formulas of 
the Kalman filter, see Harvey (1989) or Durbin and 
Koopman (2001). The Kalman filter recursion assumes that 
the fitted state space model is the truth. As a result the 
standard errors for the filtered estimates do not reflect the 
additional variation induced by the use of likelihood 
estimates for the variance components in the state space 
model and are therefore too optimistic. 
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Figure 5.8 Filtered estimates and GREG estimates based on all 
waves for the monthly unemployment rate 
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Figure 5.9 GREG estimates based on the CAPI wave and based 
on all waves for the monthly unemployment rate 
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Figure 5.10 Standard errors of the GREG and filtered estimates 
for the monthly unemployment rate 
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As expected, the standard errors of the GREG estimates 
based on all waves are smaller than the standard errors of 
the GREG estimates based on the CAPI wave, since they 
are based on more data. The standard errors of the filtered 
estimates are smaller than the GREG estimates based on all 
waves, since the time series model uses additional sample 
information from preceding periods. The standard errors of 
the filtered estimates are slightly but continuously de- 
creasing during the period 2003 to 2008. 

The size and complexity of the applied time series model, 
is large compared to the length of the series available to fit 
the model. The final model that is applied to a five 
dimensional series which is monthly observed during a 
period of eight years contains 41 state variables. Therefore it 
is worthwhile to consider more parsimonious models, which 
might reduce the standard errors of the filtered estimates. 
Furthermore, the GREG estimate contains a bias since the 
RGB contains a seasonal effect, which is not reflected by its 
standard error. Therefore, the efficiency obtained by 
borrowing sample information from the past by relying on a 
time series model is illustrated more clearly if the standard 
error of the GREG estimates using all waves is compared 
with the standard error obtained with a time series model 
that accounts for the RGB in the trend only. Therefore a 
time series model without a component for the RGB in the 
seasonal pattern is applied to the data to illustrate the 
variance reduction by borrowing strength over time. The 
filtered estimates for the monthly unemployment rates based 
on a model with and without a component for the RGB in 
the seasonal pattern are compared in Figure 5.11. 
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Figure 5.11 Filtered estimates of the monthly unemployment rate 
for two different time series models 


The model without a component for the RGB of the 
seasonal effects assumes a seasonal effect for the population 
parameter 9, that is based on an average of the seasonal 
effects of the five waves. The absolute values of the 
seasonal effects in February, March, and August are smaller 
under the simplified model, resulting in a lower estimate for 
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the monthly unemployment rate in February and March and 
a larger estimate in August. This results in a more 
pronounced seasonal pattern in the filtered series obtained 
with the complete model. 

The standard errors of the filtered estimates obtained 
with the two time series models and the standard errors of 
the GREG estimates using all waves are compared in Figure 
5.12. The standard error of the filtered estimates of the 
simplified time series model is substantially smaller than the 
standard error of the GREG estimates using all waves. The 
simplification of the time series model by ignoring the RGB 
for the seasonal effects, results in a further reduction of the 
standard error at the cost of an increased bias in the seasonal 
effects. Under the model assumption that the estimates 
based on the first wave are unbiased, the time series model 
that accounts for the RGB in the seasonal effects is 
preferred, since it removes the bias in the seasonal pattern. 
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Figure 5.12 Standard errors of the GREG estimates based on all 
waves and filtered estimates for two different time 
series models for the monthly unemployment rate 


Discussion 


In this paper a multivariate structural time series model is 
applied to the monthly data of the LFS that accounts for the 
rotating panel design of this survey. This approach is 
initially proposed by Pfeffermann (1991) and extended in 
this paper with a component that models systematic 
differences in the seasonal effects between the subsequent 
waves. Compared with the GREG estimator, which is 
currently applied in the regular LFS, the time series model 
results in a substantial increase of the accuracy of the 
estimates of the unemployment rate. Firstly, the model 
explicitly estimates the RGB in the trend and the seasonal 
patterns between the first CAPI wave and the four 
subsequent CATI waves. Secondly, the time series model 
borrows strength from data observed in preceding periods 
via the assumed model for the population parameter and the 
AC between the survey errors of the different panels. 
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The RGB induced by the rotating panel design is 
substantial. The bias in the trend results in an under- 
estimation of the unemployment rate in the subsequent 
waves and its magnitude slightly decreases from -0.8 
percent points in the second wave to -1.1 percent points in 
the fifth wave. The seasonal patterns of the first two waves 
and the last three waves are also significantly different, 
since the seasonal pattern in the last three waves is less 
pronounced. 

A parsimonious time series model that accounts for the 
RGB in the trend but not for the RGB in the seasonal 
pattern, results in a further reduction of the standard error of 
the filtered estimates. This, however, results in a biased 
seasonal pattern in the monthly estimates of the unemploy- 
ment rates. Since the standard errors of the filtered estimates 
obtained under this parsimonious model do not reflect this 
bias, a time series model that accounts for both the RGB in 
the trend and the seasonal pattern is preferred. 

The time series model is identified by adopting a 
restriction for the RGB parameters which assumes that the 
first wave is observed without bias. This implies that the 
estimates based on the first wave are used to benchmark the 
subsequent waves. If this restriction is used, then an all out 
effort in each part of the statistical process is required to 
reduce possible bias in the first wave, e.g., by using the most 
appropriate data collection mode, reducing nonresponse, 
optimizing the weighting scheme, efc. Based on external 
information about the bias in the different waves, the 
restrictions for the RGB components might be adjusted. 

The time series approach explored in this paper is 
appropriate to produce model-based estimates for monthly 
unemployment figures. Statistics Netherlands, however, is 
generally rather reserved in the application of model-based 
estimation procedures for the production of official 
statistics. Model misspecification might result in severely 
biased estimates. This bias is not reflected in the standard 
errors of the Kalman filter estimates. Extensive model 
selection and evaluation is therefore required for each 
separate target variable. This hampers a straightforward 
application of such estimation techniques, since there is 
generally limited time available for the analysis phase of the 
regular production process of official releases. 

There is, on the other hand, a case for having official 
series that are based on model-based procedures with 
appropriate methodology and quality descriptions for 
situations where direct estimators do not result in 
sufficiently reliable estimates. The RGB observed under the 
rotating panel design of the LFS clearly illustrates the 
existence of non-sampling errors such as measurement 
errors and panel attrition. Therefore the traditional concepts 
that observations obtained from sampling units are true 
fixed values observed without error and that the respondents 
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can be considered as a representative probability sample 
from the target population, generally assumed in design- 
based sampling theory, are not tenable under such designs. 
The application of direct estimators in the case of 
measurement errors and selective panel attrition will result 
in severely biased estimates. In the regular estimation 
procedure a ratio correction is applied to the GREG 
estimates, which is based on the implicit model assumption 
that the bias is constant over a period of three years. The 
time series model applied in this paper can be used to 
produce estimates that are corrected for the bias introduced 
by these non-sampling errors in a more advanced way. 

This estimation procedure is also applicable in situations 
where small sample sizes result in unacceptable large 
standard errors. Small sample sizes arise if official statistics 
are required for small domains or for short data collection 
periods like the monthly unemployment figures in the LFS. 
Most surveys conducted by national statistical institutes 
operate continuously in time and are based on cross- 
sectional or rotating panel designs. Consequently, estima- 
tion procedures based on time series models that use sample 
information observed in preceding periods are particularly 
interesting. 

The time series model yields estimates for the trend and 
seasonal components of the population parameter. Season- 
ally adjusted parameter estimates and their estimation errors 
are therefore obtained as a by-product of this estimation 
procedure. Another major advantage is that this approach 
accounts for the AC in the survey errors due to the rotating 
panel design. Pfeffermann ef a/. (1998) show that ignoring 
these AC, for example with the Henderson filters in X-12- 
ARIMA (Findley, Monsell, Bell, Otto and Chen 1998), 
results in spurious trend estimates. 

The model can be improved in several ways. Information 
about registered unemployment and related variables, 
available in the register of the Office for Employment and 
Income, can be used as auxiliary variables in the models. If 
longer series become available, an additional cyclic compo- 
nent might be required to capture economic fluctuations. 
Another possible improvement is detection and modelling 
of outliers. Furthermore the model needs to be extended to 
estimate monthly unemployment rates for different domains 
using sample information collected in the past as well as 
cross-sectional data from other small areas, using the 
approach proposed by Pfeffermann and Burck (1990) and 
Pfeffermann and Tiller (2006). 
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Estimates for small area compositions subjected 
to informative missing data 


Li-Chun Zhang ' 


Abstract 


Estimation of small area (or domain) compositions may suffer from informative missing data, if the probability of missing 
varies across the categories of interest as well as the small areas. We develop a double mixed modeling approach that 
combines a random effects mixed model for the underlying complete data with a random effects mixed model of the 
differential missing-data mechanism. The effect of sampling design can be incorporated through a quasi-likelihood sampling 
model. The associated conditional mean squared error of prediction is approximated in terms of a three-part decomposition, 
corresponding to a naive prediction variance, a positive correction that accounts for the hypothetical parameter estimation 
uncertainty based on the latent complete data, and another positive correction for the extra variation due to the missing data. 
We illustrate our approach with an application to the estimation of Municipality household compositions based on the 
Norwegian register household data, which suffer from informative under-registration of the dwelling identity number. 


Key Words: Conditional MSE of prediction; EMPQL algorithm; Generalized SPREE; Not missing-at-random; Two- 


way contingency table. 


1. Introduction 


Small area (or domain) population counts cross-classified 
by various social-economic characteristics are increasingly 
demanded for fund allocation, regional planning and social- 
economic research. Purcell and Kish (1980) outlined the so- 
called “Structure preserving estimation” (SPREE), which 
operates by modifying the small area estimates in a way so 
that they vary from one area to another in accordance with 
the variation that exists in another known auxiliary table of 
the same dimension. Typically the auxiliary table is obtained 
from a previous census, or some administrative register 
containing similar information. Zhang and Chambers (2004) 
developed a generalized SPREE (GSPREE) approach. Both 
fixed effects and random effects mixed models were 
introduced, and the restricted log-linear model underlying 
SPREE was shown to be a special case. This provides means 
for reducing the potential bias of the traditional SPREE 
estimates. We refer to Ghosh, Natarajan, Stroud and Carlin 
(1998) and Longford (1999) for alternative hierarchical and 
empirical Bayes approaches to this type of data. 

In this paper we extend the GSPREE approach to 
situations subjected to missing data. This can be useful in 
sample surveys where nonresponse is unavoidable. We 
concentrate on small area compositions that can be arranged 
in a two-way table, where one of the two dimensions refers 
to the small areas and the other refers to the categories of 
interest. The cell counts summarize to a fixed area total that 
may or may not be known. For instance, each person 
between 16 and 74 years of age can be classified according 
to the labour force status “employed”, “unemployed” and 
“not in the labour force”. The sum of the three counts inside 


a small area is the total number of persons between 16 and 
74 years of age within this area. 

In the context of small area composition we say that the 
missing-data mechanism is informative provided it varies 
across the categories of interest. As such it is also not 
missing-at-random (Rubin 1976). In addition, the overall 
rate of missing differs across the areas. Differential 
missingness as such leads to distortion of the underlying 
complete data, and bias if the estimation is carried out as if 
the observed data were complete. We propose a double 
mixed modeling approach that combines the random effects 
mixed model for the underlying complete data with a 
random effects mixed model of the missing-data 
mechanism. The double-smoothing approach is outlined in 
Section 2. 

It should be noted that national statistical offices that 
conduct large scale surveys will have accounted for missing 
data by weighting adjustments or imputation. This, how- 
ever, will have been done at levels that are significantly 
higher than the small areas, and will be for variables that do 
not necessarily correspond to those of interest for the small 
areas. When available, the adjusted totals can be 
incorporated into the GSPREE as marginal totals for 
iterative proportional fitting (IPF). But modeling of the 
differential probabilities of missing across the small areas 
will generally remain a matter of interest. 

It should also be noticed that informative missing data as 
such makes it less straightforward to assess the potential 
bias of any estimation approach. SPREE may be biased on 
two accounts: (i) the underlying restricted log-linear 
assumptions are likely to be unrealistic, (ii) direct IPF may 
fail to account for the differential probabilities of missing 
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adequately. The proposed double mixed modeling approach 
deals with problem (i) by GSPREE modeling of the 
underlying complete data, and it deals with problem (11) by 
introducing a more flexible missing-data model, as we shall 
discuss in Section 2.2. Nevertheless, bias is likely to persist 
to a certain extent. Since the estimation of model parameters 
and random effects is more complicated under the double 
mixed modeling approach, alternative estimation methods 
that are able to preserve the computational simplicity of 
SPREE, while making more adequate adjustment for 
informative missing data, are worth investigating in future. 

When it comes to the assessment of estimation un- 
certainty, Booth and Hobert (1998) argued for the 
conditional mean squared error of prediction (CMSEP) 
given the observed data. We extend their approach and 
derive approximate CMSEP in the current multivariate 
incomplete-data situation. This results in a_ three-part 
decomposition of the CMSEP, corresponding to a naive 
prediction variance, a positive correction that accounts for 
the hypothetical estimation uncertainty of the parameters 
based on the latent complete data, and another positive 
correction for the extra variation due to the missing data. 
The details are given in Section 3. 

Estimation procedures for the parameters, the CMSEP 
and the small area compositions are described in Section 4. 
In Section 5 we apply our approach to derive estimates of 
the Municipality household compositions based on the 
Norwegian household register, which suffers from infor- 
mative under-registration of the dwelling identity number 
(DIN). A summary is given in Section 6. 


2. Double mixed modeling 


2.1 Random effects mixed model in the complete- 
data case 


2.1.1 Models for finite population 


The small area counts can be arranged in a two-way 
contingency table, denoted by X={X,,}, where 
a=1,..., A indexes the small areas and k =1,...,K the 
categories of interest. The interest of estimation is the 
within-area proportions given by 


K 
Oe sce as ele of UX 


— 
I 


referred to as compositions since ¥,0%, =1. Typically 
under the GSPREE approach we assume that the marginal 
totals {X,} and {X,}, also known as the allocation 
structure, are either known or can be reliably estimated, in 
which case estimating {6*,} is equivalent to estimating 
{X_,}. For simplicity we then make no distinction between 
counts and compositions in the exposition. Otherwise, 
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without the allocation structure, one can still use our 
approach to estimate {0*,} but not {X,}. 

Assume that we have available an auxiliary table of the 
same dimension, denoted by X°={X%}, and the 
corresponding within-area proportions {6°,}. To model 
0* =(0%,,...,02,)’ we use the multinomial standardized- 
log (mslog) link function, given by 


K 
p32, = log 64, — K ">? log 07, (1) 
j=l 
and similarly for 1°, and 6°,. Zhang and Chambers (2004) 
introduced the following generalized linear structural mixed 
model (GLSMM) 


Ck =A, + Bue, Wig 
where (2) 


K K 
ae S04 andi a= 
k=l 


fae 


_ 


and Vig) = (Yao -++> Vax )’ assumes a multivariate normal 
distribution with covariance matrix G=G(65), where 6 
contains the variance parameters. Notice that there is no 
area-specific term in (2) because ,p1,, =d,Ho, =0. The 
term “structural” refers to the fact that this is a model of the 
finite-population parameters {0%,} directly, although the 
emphasis is not common in the small area estimation 
literature. For instance, the well-known Fay-Herriot model 
(Fay and Herriot 1979) is “structural” in the same sense. 

There is an important interpretation of the model (2) in 
terms of the log-linear interactions of {@,,} due to the 
choice of the link function (1), i.e., 


ieecpaneeae's 3) 


where by the standard theory of log-linear models (e.g., 
Agresti 2002), we have 


log X,, =log X, +log6%, =a¢ +a* +a}; +7, 


i eye ag fire ae and ai a = KUL foo A 
G,. .and = A- nels Xu = Cnn enh ee 
log X,, -a2 i =O, ,._ such. that, Oe cas) ee = 
07, =r, a2, =0. We refer to (3) as the log-linear 
identity, and we refer to the log-linear parameters a7, as 
the (first-order) interactions of the compositions 6%, as well 
as the counts X.,. Similar identity holds for p2,. Zhang 
and Chambers (2004) showed that the GLSMM is 
equivalent to the following proportional interactions mixed 
model (PIMM) 


az, = Bay, +v, +0,(4). (4) 


The parameters A,’s in (2) do not entail any model 
restriction beyond the PIMM, and they do not affect the 


Survey Methodology, December 2009 


interactions. The parameter f is called the proportionality 
coefficient. Clearly, SPREE based directly on the asso- 
ciation structure {X°} amounts to setting B=1 and 
Vx =0. We therefore refer to the model (2) as a GSPREE 
model, which contain both fixed and random effects 
extensions of the SPREE model. 


2.1.2 Model for sample 


To complete the model specification we assume sample 
classifications x = {x,,}. Let 


Ci A yim eee Tae) 


be such that F(¢,,| v) = E(t,| X)= Oz», where v= {v,,}. 
The expectation is typically with respect to the sampling 
design. However, it can also be taken under a suitable model 
of the sampling distribution, such as a multinomial model 
for x, provided simple random sampling within each area. 
We therefore make no distinction in the notation. 

We assume that t, is independent of t,, for a#a’', and 
put 


V(ty) a V;O, (X,) and Cov(t,,, ty) aa Vi OG (X,) (5) 


where @,(-) and @,,(-) are specified variance and 
covariance functions, and v, is the dispersion parameter 
that may or may not be known. This is essentially the quasi- 
likelihood set-up for dependent data (McCullagh and Nelder 
1989). The dependence on X, allows us to incorporate the 
sampling design effect, in which case the expectations in (5) 
may be evaluated with respect to the sampling distribution. 
This is an important reason why we do not directly assume 
that the distribution of t, belongs to the exponential family, 
as e.g., in the generalized linear mixed models (Breslow and 
Clayton 1993). 


2.1.3 Parameter estimation 


Zhang and Chambers (2004) outline an_ iterative 
weighted least square (IWLS) algorithm for the GLSMM 
(2), which is a variation of the PQL approach (Schall 1991; 
Breslow and Clayton 1993). Let p, =(u4,...,u2,)’. The 
GLSMM (2) can formally be given by 


H, = 2(9,) = eG + BY) 


where g(8,) is the mslog link function, and C= 
Oar oer Dade and Verne TA totes (VOR lox The KxK 
design matrix H, and Kx(K-—1) design matrix B are, 
respectively, 

=lk4 | 

Te xK-I 


Heth e.| ands B= 
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where 1 is a vector of | and / is an identity matrix. Define 
the working variables 


def. 
Z, =H, +e, =H,¢+ Bv,+e, and e, =QO(t, — 0%) (6) 


where Q=0y*/00* is the Jacobian matrix of partial 
derivatives. Denote by R, the conditional covariance 
matrix of t, given 0° defined by (5). Under the PQL 
approach we assume that e, has an approximate multi- 
variate normal distribution with covariance matrix ORO’, 
and apply standard methods for linear mixed models 
(LMM) to the linearized data (6). Variants of the PQL 
approach differ in the estimation of the variance parameters 
6. The details are omitted here. 


2.1.4 On model hierarchy 


The GLSMM (2) is specified at the finite population level. 
More generally, we may consider the finite population {X,, } 
to be randomly generated from an infinite super-population. 
Let 6,, be the within-area probability that a unit of the super- 
population belongs to the cell (a,k), where >,9,, =1. 
Conditional on X, =>,X,,, the within-area counts 
(Xa1y-++> X,x)’ follow the multinomial distribution with 
parameters (0,,,...,9,,)'. A multinomial standardized-log 
mixed model (MSLMM) of {8,,} is given by 


May =A, + Bue, + VS 
where (7) 


K K 
any == ee ETE 


k= k=] 


_ 


where i, 1s given by 0, through the mslog link function. 

Unlike the GLSMM (2), the equation (7) defines a 
regression model. There are then three different hierarchy 
one may choose from in the sample survey situation: 


1. Assume the GLSMM (2) for the finite population 
and the quasi-likelihood model (5) for the sample, 
yielding the GSPREE approach of Zhang and 
Chambers (2004). 


2. Assume the MSLMM (7) for the super-population 
and model sample data t, based on 0, directly, 
yielding a purely model-based two-level approach. 


3. Assume the MSLMM (7) for the super-population, 
and assume that the finite population totals X, 
follow the multinomial distribution given 0,, and 
assume the quasi-likelihood model (5) given X,, 
yielding a general three-level model. 


Provided the finite population is large, it makes little 
difference in practice to adopt the GSPREE approach, in 


Statistics Canada, Catalogue No. 12-001-X 


194 Zhang: Estimates for small area compositions subjected to informative missing data 


which case one does not have to deal explicitly with one 
extra level of hierarchy. But the distinction between (2) and 
(7) becomes necessary if the areas are so small that the 
stochastic variation in X, is not negligible compared to the 
sampling variation in x, (or t,). In our application later, 
we have register data that would have given us the 
interested population counts {X,,} had they not suffered 
from missing data. And the small area level of aggregation 
is so detailed that the stochastic variation in X, can not be 
ignored. We therefore adapt the GSPREE approach by (a) 
adopting the MSLMM (7) instead of the GLSMM (2), and 
(b) modeling X, as a ‘sample’, albeit a very large one, 
from the super-population directly. 


2.2. A random effects mixed model of missing data 


Missing data add another level of stochastic variation on 
top of the underlying complete data. In the exposition 
below, we consider the sample counts {x,,} as the 
complete data, which is the most common situation in 
practice. Our application later in Section 5 can be viewed as 
a special case where X = x. 

Denote by y, =(V,)---> Vax) the observed cell counts, 
for a=1,..., A. Suppose that, conditional on x,, and a 
random effect 5, 


E( Van | Xan 00g) al X ak Pak 
and (8) 
V Vir | ba 0) = V2Cak Pak (1 a Pex) 


where c,, is a known constant, and v, is the dispersion 
parameter. We assume that y,, is independent of y,, for 
k # j, i.e., missing data are independent from one cell to 
another. Let the units in the complete sample cell (a, k) be 
indexed ibywi'=ilhy. 207), aeKetaw; gai) ifsthes i auniteis 
observed, and 7; ,, =0 if it is missing. The parameter p,, 
is the assumed probability of 7; ,, =1 inside cell (a,k). To 
see this, let x, ,, be the contribution of the i unit to x,,, 
iC, Xa = Diti%,ax> Such that yy = LiATak%iax and 


I 


E (Vail %ak> +++ Xn,,ak9Oa) = 
Nok Nak 
ye Xi ak EY, ak Aa Oh ae =1|b,) = Xan Pak: 
i=l i=l 


Notice that p,, does not depend on the value of x; ,,, but 
only the position of the unit in the two-way table. We 
assume that p,, depends on b, through the logistic link 
function given by 


Nak = 108(Pox/C - Par )) = & + 8, 
where (9) 


b, ~ N(0, 0°). 
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The fixed effects &, ’s allow the probability of missing to 
depend on the categories of interest, the area-level random 
effect b, allows it to vary across the areas in addition. 

Obviously, under the assumptions (8) and (9), the 
missing data cause bias in the estimates of the 1, ’s, if the 
observed table y is treated as if it were complete. Moreover, 
it distorts the estimation of the first-order interactions 
{a7,}. We have, 


log Pak = (& DR) mad Gk where tee = log(1 ss exp(&, +b, )): 


The first-order interactions of {p,,} are then given by 
O= Vn -Va ele oy pty Ob Ulcestow mane 
column means 7, and 7, and the overall mean y . These 
are non-zero unless €, =€ By (8) the interactions of the 
expected observed table are given by 
Oe) = Oy + OF, = Oe — Yay FO 

such that the estimates of {a4} will be biased if y is treated 
as X. 

It is worth noting that, as far as the estimation of the 
interactions is concerned, it is in principle possible to treat 
the observed table y as if it were the complete table x 
under a particular missing-data model given by 


log p,, =€, +B. (10) 


This is because the first-order interactions of {p,,} are all 
zero under (10), in which case we have a8!" =a¥,. 
Disregarding the range restrictions, the assumption (10) 
defines an informative missing-data mechanism where the 
probability of missing varies across the categories of 
interest, while the area effect modifies all the within-area 
probabilities of missing by a factor exp(b’), such that 
Pax! DXj=1 Pa; = eXP(E,)/X,exp(G;,) remains constant. The 
model (9), however, is more flexible since it allows the 
random effects to affect the interactions. Both (9) and (10) 
will be examined in Section 5. 

Finally, we notice that allowing for component-wise 
random effects in the model (9) may cause identification 
problems. For instance, assume simple random sampling 
from the finite population, in which case the interactions of 
the expected complete table are given by ak =a2. 
With component-wise 5, in the model (9) we have 
log Par = Sk + Oak +Yar> Where y,, =log(1+exp(§, +5,,))- 
It follows from (4) and (8) that the interactions of the 
expected table E(y|x,b) is given by Bao, +v, + 
by —Yqx- But there is no information in the observed data 
to distinguish between the two random effects v,, and b,. 
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3. Conditional mean squared errors of prediction 


We adopt the approach of Booth and Hobert (1998) and 
use the CMSEP as a measure of the uncertainty in 
prediction. Like them we consider the CMSEP on the 
linear-predictor scale. In vector form the 1, ’s in (1) belong 
to the following class of linear functions 


w, =H,6+ BV. (11) 


where 1, is the area-specific vector of linear predictors, 
and ¢ is the vector of fixed effects, and v, is the vector of 
area-specific random effects, and H, and B, are the 
corresponding design matrices. All the quantities have been 
specified in (6) for the GLSMM (2), where we actually have 
B,=B8. But we shall adopt the slightly more general 


formulation (11) in the following. Let ¢ and VW, be,. 


respectively, the estimates of ¢ and v, based on obser- 
vations subjected to missing data, denoted by y, for 
a=1l,...,A. The CMSEP of f,=H,C+8,¥, is defined 
as 


CMSEP, = E{(h,-b,) (i, — ba)” | hese 


We introduce first a decomposition through the 
hypothetical best predictor (BP) based on x,, given by 
= E(u,|x,, 6,5) =H,¢+ B,E(v,|x,,¢,5), when the 
parameters are known. We have 


CMSEP, = E{E((h, Ha Mga)’ Xa) 1¥a} 
+E{(A,—p,)0,-0,) ya} 
= E{B,Cov(¥,, Va |Xa)Bz la} 
FEA pia) |¥ah 


because j1,—p, and fi, —[1, are conditionally independent 
of each other given x,: {1—p depends on the random 
effects v,, whereas ,i,—,1, depends on random variations 
in the other areas. Next, for the second term on the right- 
hand side, we introduce a decomposition through the 
hypothetical estimated best predictor (EBP) based on the 
complete data x, denoted by i =H ,C+ Bi where 
(c,d) are the parameter estimates based on x, and 
i, = E(v,|x,, 6,5). We have 


E{(a,-1,)(@,-0,)' ly} * H{G,-0,)(0,-6,) } 
= E{E(0,-1,),-0,)' |x)} 
+ E{E((A,-f,)(0,-f,)' |x)} 
= E{(i,- ph, - f,)"} 


+ E{E(0,-f,h,-f1,)° |x)}- 
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The first approximation is correct to the order of Ora” ), 
and can be justified as the number of areas tends to infinity. 
Intuitively, this makes sense if the information from any 
single area is asymptotically negligible compared to the 
information from all the other areas together. Next, the 
decomposition follows because fi,—j1, and fi,—fi, are 
independent of each other given x: the former is a constant 
given X. 

In this way, we obtain an approximate CMSEP with a 
three-part decomposition 


CMSEP, * hy (X43 6, 8) + hag (6,8) + fy. (x36, 8, w) 


where w contains the parameters of the conditional 
distribution of y, given x,, and 


def. 
h,,(x,; 6,5) = BCov(v,, v,|x,)B, (12) 


def. 


hy, (68) = E{G,- 2G) =a") (13) 


ats def. 
hy, (x5 6,5, y) = E{(i,-f,0,-f8,)" |x}. (14) 


The three h-terms correspond, respectively, to a conditional 
prediction variance due to the random effects, a positive 
correction that accounts for the uncertainty in the estimation 
of the parameters based on the latent complete data, i.e., the 
sampling variation, and another positive correction for the 
extra variation due to the randomness in the missing data. 
Alternative approximations are possible. For instance, one 
might use E{B,Cov(v,, v, |X,)B. |y,} instead of h,,, or 
replace h,, with the unconditional E£{(i,-{,) 
(= fa \. The expressions (12) - (14) are chosen 
because they produce a clean separation between the 
sampling variation in the complete data and the extra 
variation owing to the missingness given the complete data. 
The difference from the CMSEP in the complete-data case 
(Booth and Hobert 1998) comes down to the third term 4, 


4. Estimation 


4.1 Parameter estimation 


The structure of the data suggests an iterative procedure 
similar to the EM algorithm (Dempster, Laird and Rubin 
1977). Given the current values of the parameters and the 
random effects, we calculate at the E-step the conditional 
expected complete two-way table E(x|y,m). At the M- 
step we estimate the two random effects mixed models 
separately by some maximum penalized quasi-likelihood 
(MPQL) procedures. Iterations between the two yield an 
EMPQL algorithm. 
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For the E-step, let J; ,, =1 if the sample unit 7 belongs to 
the (a,k)" cell, and J, ,, =0 otherwise. The value is 
observed provided 7, ,, =1, but is unknown if 7; ,, =0. Let 
8, be the generic compositions, depending of the adopted 
model. Suppose that 


PU, 4 =1lies]= Ph NS 


where s denotes the complete sample, and d,, is some 
known constant which accounts for the eventual sampling 
design effect. For example, simple random sampling implies 
that d,, =1 for all (a,k). An example of d,, #1 is when 
the sampling units are households, which are selected by a 
probability proportional to the household size. Let m,, = 
Xak — Yak = Lisr,ae=04;,at%i,ak- We have E(x,|y,.m,.) = 
y,+£(m, | m.,), where 


E(m,,|m, )= DS EQ; ak \%, ak = 9%, ak 


Bh ak =0 


= mpPUlin = l | Ti. Gk = 0] 


=m, (1 Pax dau Pau 2 (1 ~ Pa )da8 (15) 


Having thus ‘completed’ the sample data, we move to the 
MPQL-step, where we apply the IWLS algorithm outlined 
in Section 2.1.3, respectively, to the complete-data model 
and the missing-data model conditional on the complete 
data. 


4.2 Estimation of CMSEP 


Evaluating the CMSEP at the estimated parameter values 
yields a plug-in estimate of the CMSEP. Of the three h- 
terms, h,, is of the order O, (1), whereas both h,, and h,, 
are of the order O, (A'), when the number of areas tends 
to infinity while the within-area sample sizes remain 
bounded. The results of Booth and Hobert (1998) and 
Prasad and Rao (1990), obtained in the univariate complete- 
data case, suggest that the bias in the plug-in estimate h,, is 
of the same order as hy, and h,,,- These authors developed 
second-order correction through the Taylor expansion. We 
do not pursue such second-order asymptotics in this paper. 
Approximate expressions of the -terms that accompany the 
EMPQL algorithm are given below. 

Take first h,, by (12). Based on the linearized data (6), 
the covariance matrix Cov(v,, v,|Z,) does not depend on 
either z, or x,. This is convenient because we then have 


h,,(X,; 6, 5) ¥ B,Cov(v,, V,|z,)B. 
=BYUG'—GB/V2 BYG)YB? 2 (16) 
where V, =B,GB’ +QR,Q" is the marginal covariance 


matrix of z,. 
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Next, take h,, by (13). Let o=(C’,8")’. Expanding $ 
around @ yields f,-p,*1,(>—), where [1 = 0f1,/00, 
such that 


h,, = ft, Cov(d, >) hy. (17) 


Based on (6) we derive ft,=H,C¢+D,u., where 
D,=B,GB'V;' and u,=z,-H,¢. Denote by J the 
identity matrix. The partial derivatives in ji’, are given by 


Ofi,/06 =(1 - D,)H, 
and 
Oj1,/08, = (@D,/08, a, = (I - D,)B, (0G/08 ,) BV, 


where 6, is the j" variance parameter in the covariance 
matrix G(5) of v,. To obtain Cov(9, >), suppose that the 
PQL approach is based on the following quasi log- 
likelihood 


t=> 2, 


and 
l= -Slog Wa I-5@, -H,ty Vauze = FTG); 


The so-called sandwich formula yields then 


~> ( @e) [Afae,\(ae,¥l( ae) 
conte =F alee a he ieee ? 


Finally, take h,, by (14). Similarly as above we have 
i,=U-D,)H,6+D,z, evaluated at @ = o, and fi,= 
(1-D,)H,6+D,2,, where Z, is derived from t, = 
t(x,) for &, = E(x,|y,,m,;, ¥). Expanding around 
o and retain only the leading term, we obtain 

te =e ~ LL, =e = NZ. —Z,) 
where fi, =(1-D,)H,6+D,zZ,, and Z, is derived from 
t, =t(X,) for x, = E(x,|y,,™m,;, V). That is, we ignore 
the terms involving -—. The remaining variation in Z, is 


due to the estimation of the missing-data model alone. 
Expanding W around w, we obtain, by the chain rule, 


h,, = C,Cov(y, |x) Cz 
and (18) 


<a SSSI. 


where we assume that E(W|x)=wy and £[z,|x]=Z,. 
Whereas the sandwich formula yields Cov(W, |x) under 
the conditional model of y given x, similarly to Cov(d, 6) 
above. 
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4.3 Estimation of small area compositions 


Suppose first that the GLSMM, defined by (2) and in 
combination with (5), has been estimated, upon which we 
obtain fi, and 0%, = exp(fi;,)/E, exp((i3,). 

When the marginal totals X, and X, are known, it 
makes sense to apply the IPF, starting with the estimated 
table {6%}. The difference from SPREE, which starts with 
the auxiliary table X°, is that the interactions have been re- 
estimated. On convergence we obtain the estimated small 
area counts, denoted by X = {X a }» and the corresponding 
compositions, denoted by 03, =X,,/Xj,X,, which are 
different from the direct model estimates 6%, that have 
provided the starting values for the IPF. 

Often in practice, while the area totals {X,} may be 
known, the marginal totals {X,} need to be estimated 
based on the survey data available, separately using a 
method that is appropriate for the aggregated level. The IPF 
is still worth considering as long as these estimated marginal 
totals are judged to be more reliable and/or less biased than 
the aggregated small area estimates )>, X nee The reason 
is that the estimated interactions &7, are preserved in the 
IPF, ie., a7, =a%. By the log-linear identity (3), the 
difference between the direct model estimate 6% and final 
estimate 0+, is due to the difference in the estimates of the 
main effects {a*}. Thus, less biased estimates of {X,} 
are expected to yield less biased estimates of {a?} and, 
thereby, less biased estimates of {0+ }. 

Suppose next that the MSLMM (7) combined with (5) 
have been estimated. We may express the interest of 
estimation, ie., {u7,}, in terms of z, defined as 


z,=H,6+B,v, +e, =H,C+B,v,+e +e%* 


= px +e7X = HOC+ v* + e" 


where e* =Q(0*-6,) and e%* =Q(t,-07). In 
accordance we have R,=R*+R**, where R* = 
Cov(6*,0* |.) and R** =Cov(t,,t, |92). It follows 
that 


py =H, + (B,GB, + ORS O")V,'@, - 6). (19) 
The rest follows as above where 12 is estimated directly 
under the GLSMM. 
5. Example: Register-based small area 
household compositions 
5.1 Register household data 


Register-based household data have undergone consid- 
erable development in Norway. One of the goals is to 
produce detailed household statistics that traditionally are 
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only available from the census. For this purpose the 
registration of a unique dwelling identity number (DIN) was 
initiated in the last census in 2001. The work is not yet 
completed, and the DIN is still missing for about 6% of the 
people residing in the country. The rate of missing is 
differential as it varies over the household type as well as 
across the Municipalities, the latter of which is a reflection 
of the overall effort of the local administration regarding the 
registration of the DINs. 

A household register can be complied in a year after the 
census based on a number of data sources. The most 
important ones include the central population register 
(CPR), the DIN-register and the census household file 
(CHO1). Even without the DIN a register household can be 
compiled based on the other information available. But the 
result suffers from informative under-registration of the 
DIN. For instance, a typical source of bias is cohabitants 
living without children, because such a couple appear as two 
single-person households in the CPR, unless they have 
already been identified as a household in the CHO1. 
Nevertheless, historic as well as cross-country comparisons 
suggest that the national totals are acceptable. A more 
urgent problem lies on lower levels of aggregation. For 
example, changes from the census in 2001 are unlikely large 
in certain Municipalities, including the capital city Oslo 
where the increase in the proportion of single-person 
households is almost three times as high as it is in the rest of 
the country - see top-left plot in Figure 1. And a large part of 
the problem in Oslo can be explained by a combination of 
high proportion of cohabitants living without children and 
low DIN-registration rate (indeed, the lowest in the 


country). 


5.2 Set-up of data 


We shall illustrate our approach using these register 
household data. The target population contains all persons 
living at multiple-dwelling addresses at the beginning of 
year 2005, who do not belong to households of married 
people or registered partners; the latter household types are 
excluded because the DIN is not critical for compiling the 
households of these people. There is no distinction between 
the finite population and the sample in this case, i.e., X = x. 
The households that have registered DINs are treated as the 
‘observed’ sample y, whereas the households that do not 
have registered DINs are viewed as the missing. In this way 
the population consists of 713,387 persons, of which 
558,136 persons have registered DINs. The overall rate of 
missing 1s about 22%. 
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Difference between estimates of proportion of Single-person households and census counts in 2001 against log 


Municipality size: Register households (top-left), Households with registered DINs (top-right), SPREE based on 
census (middle-left), DirSPREE based on households with registered DINs (middle-right), SupGSPREE of 
super-population proportions (bottom-left), and ImpGRSREE of imputed finite-population proportions 


(bottom-right). The dashed line marks no difference 


Let the Municipalities be the small areas of this study, 
where A=433. The households are classified into 4 
categories: k =1 for “Single-person”, k=2 for “Single- 
parent”, k=3 for “Cohabitants”, and k=4 for “Other”, 
i.e., K =4. Let i index the households, and let x, be the 
number of persons living in the household. Let X,,, = x,, 
be the number of persons in the (a,k)" cell in the 
population, and let y,, be the corresponding ‘observed’ cell 
count. Let N, be the number of households in the (a, k)" 
cell, and let 7,, be the corresponding number of ‘observed’ 
households. Notice that only the total number of persons is 
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known in each area, but not the total number of households. 
However, provided cell-specific probability of DIN- 
registrations, an estimator of NV, based on X,,, is given by 
N yp =M%yX qp/¥qx- We shall therefore concentrate on the 
estimation of X,,, here. 

Let {X°.} be the corresponding cell counts from the last 
census in 2001. Let X/, = y,, +m, be the register counts 
in 2005, where m/, is the number of persons without the 
DIN. A register household can be considered as a form of 
imputed household that may suffer from informative 
missing of DINs. The register area total is correct, i.e., 
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X/ =X,, and the national totals {X',} are considered 
acceptable. The question is whether estimates of {X,,,} can 
be derived, based on the ‘observed’ y and the allocation 
structure {X,} and {X‘}, that better accounts for the 
differential missing DINs. 


5.3 Set-up of model 


Scatter plots of the register first-order interactions fas} 
against the census interactions {a°, } provide motivation for 
the PIMM (4). To chose between the GLSMM (2) and the 
MSLMM (7), we look at the difference between the register 
proportion ee and the corresponding census proportion 
0°., ie, 0%, —-0°., plotted against log X_: the case of 
k =1 is shown in the top-left plot of Figure 1. Clearly, the 
variance of the difference increases as X, decreases, and is 
not constant of X,. Notice that we are dealing with 
estimation at a very low level of aggregation here, where 
e.g., the median value of all {X/,} is only 70. We therefore 
adopt the model (7) for @,,, the quasi-likelihood (5) for 
X 4, =X» and the quasi-likelihood (8) and the model (9) 
fOr 

For the quasi-likelihood (5) we assume v, =1. Let 
ty =AX4/Xq- We have 


V (ty ) oe NO) (l ra Ox Xie 
and 
Cov(t x, ty) Tl =N; 84, Oe Xig ee 


where ¥ = yMex?/N and X, =X,/N,. Since x, 21, 
we have X” > X th and over-dispersion compared to the 
Multinomial-(V,,9,) distribution. We calculate the factor 
X'/X" based on the register data, which is then used as 
X¥/X2_ in the estimation below. Moreover, for the quasi- 


likelihood (8) we assume v, =1, and 


V (Varl Max) = ap aed 
T=) 
% (De viat | a) = Cag =(Lxia} 


5.4 Estimation results 


Six different estimators of the proportion of Single- 
person households (i.e., for & = 1) are illustrated in Figure 1. 

To start with, we have the direct register proportions on 
in the top-left plot, and the ‘observed’ proportions 8”, in the 
top-right plot. On average the proportion is increased based 
on the entire register compared to the census in 2001, 
whereas it is slightly decreased according to the ‘observed’ 
part only. This demonstrates that the missing DINs are 
informative, as explained before. Inclusion of the register 
households without the DINs raises the proportion of 
Single-person households. But the result is implausible in 
some of the largest Municipalities. Of course, large bias also 
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exists among the smaller Municipalities, but these are not 
easily detectable in a plot like this one. 

Next, in the middle-left plot of Figure 1, estimates are 
obtained by SPREE using the census counts {X°,} as the 
starting values. For the simple two-way table here, this yields 
an almost constant adjustment of the census proportions, with 
negligible change in the between-area variation. In the 
middle-nght plot, estimates are obtained by SPREE using the 
‘observed’ table {y,,} as the starting values. Notice that, to 
start with the observed sample counts would be too unstable 
to be useful in usual survey sampling situations, but it is a 
viable option here because of the large amount of ‘observed’ 
data. To distinguish from the standard SPREE we shall refer 
to it as the direct SPREE (DirSPREE). As noted earlier, 
DirSPREE is unbiased under the assumption (10) of 
informative missingness. Indeed, it is seen to lead to useful 
adjustments for the largest Municipalities. 

In the bottom-row plots of Figure 1, estimates are 
obtained using the double-mixed modeling approach. The 
estimates of the bottom-left plot are obtained by the IPF 
starting with the estimated super-population compositions 
{6.,}> denoted by SupGSPREE. The extreme post-censal 
development in the largest Municipalities are reduced. But 
the changes from the census-proportions are clearly over- 
shrunk towards to the population average for the smaller 
areas. The variation is e.g., much less than that of 0’, — 0°, 
in the top-left plot. The estimates of the bottom-right plot 
are derived from the imputed finite-population counts, 
denoted by ImpGSPREE, which are calculated at the E-step 
of the EMPQL algorithm. The estimates for the largest 
Municipalities are similar to those of SupGSPREE, and the 
variation in the changes from the census-proportions is 
similar to that of DirSPREE. 


5.5 Estimation of CMSEP 


Approximate CMSEP of the ImpGSPREE compositions 
can be derived similarly as in Section 3. Denote by X,,, the 
ImpGSPREE count, and by X,, the BP based on known 
conditional distribution of X, given (y,,™m, ). We have 


CMSEP(X,) = E{(X, — X,)(Xa— Xa)" 1¥a a3} 
+ E{(X, —-X,)(X =X,)"}. 
Moreover, let be the hypothetical estimate of based on 
the complete data x = X, and let w be the estimate of y 
based on the observed data. Let Q, and Q, be, respectively, 


the Jacobian matrix of partial derivatives @X,/6> and 
OX,,/dy. We have 


E{(X, -X,)(X,-X,)'} 
2 E{(X,=X,\(X,-X,)'} 
+E{(X,-X,)(X, -X,)" |X} 
=~ O,Cov(>, 6) OF + O,Cov(, | X)Q}. 
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Together, these lead to a three-part decomposition of the 
CMSEP similar to (12) - (14). In the estimation of the 
CMSEP below we ignore the effect of IPF. This is justified 
in our case because the IPF essentially amounts to a 
constant multiplicative adjustment very close to unity, as 
can be seen in the middle-left plot in Figure 1. 

The CMSEP of a DirSPREE count is calculated as a 
‘sampling’ variance that is induced by missing-at-random 
within each cell of the two-way table, plus a squared bias 
term which is estimated by the squared difference between 
the ImpGSPREE count and the corresponding DirSPREE 
count, provided the assumption (9) is a more appropriate 
model for the missing data than the assumption (10). 

The estimated root CMSEPs (remsep) are given in Figure 
2. On average both are decreasing as the Municipality size 
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increases. However, for some of the largest Municipalities, 
the CMSEP of the DirSPREE proportion is abnormally 
large for Single-person and Cohabitants households due to 
the bias term. On the whole the CMSEP of the 
ImpGSPREE composition is clearly smaller than that of the 
DirSPREE. The h,,, -term, corresponding to the prediction 
variance of X_,, is by far the dominating contribution to the 
CMSEP (over 99% in many areas). This is understandable 
since there are over 550 thousand people in the ‘observed’ 
sample, such that the uncertainty in parameter estimation is 
comparatively negligible. But the quoted percentage will be 
lower in a sample survey situation, as the estimation 
uncertainty summarized in terms h,, and h,, increases. 
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Figure 2 Estimated root conditional mean squared error of prediction (rcmsep) of DirSPREE (circle) 
and ImpGSPREE (triangle) of Municipality household proportions 
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6. Summary 


In the above we outlined a double-mixed modeling 
approach that extends the GSPREE methodology to estima- 
tion of small area compositions subjected to differential 
missing data. An approximate CMSEP was derived which 
contains a three-part decomposition, corresponding to the 
prediction variance of the unknown random effect, the 
sampling variance in the absence of missing data, and the 
extra variance due to the missing data, respectively. The 
approach was applied to the Norwegian register household 
data, which yielded useful adjustments for informative 
missing of dwelling identity numbers. 
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Small area population prediction via hierarchical models 


Debora F. Souza, Fernando A.S. Moura and Helio S. Migon ' 


Abstract 


This paper proposes an approach for small area prediction based on data obtained from periodic surveys and censuses. We 
apply our approach to obtain population predictions for the municipalities not sampled in the Brazilian annual Household 
Survey (PNAD), as well as to increase the precision of the design-based estimates obtained for the sampled municipalities. 
In addition to the data provided by the PNAD, we use census demographic data from 1991 and 2000, as well as a complete 
population count conducted in 1996. Hierarchically non-structured and spatially structured growth models that gain strength 


from all the sampled municipalities are proposed and compared. 


Key Words: Markov Chain Monte Carlo (MCMC); Population projection; Spatial models. 


1. Introduction 


Like many other countries, the demand for detailed and 
updated small area statistics has been steadily growing in 
Brazil. This increasing demand is motivated by the need to 
have a more precise picture of subregions and has been 
driven by issues of distribution, equity and disparity. For 
instance, there may exist subregions or subgroups that are 
not keeping up with the overall average in certain respects. 
Therefore, there is a need to identify such regions and to 
have statistical information at that geographical level before 
taking any possible remedial action. Besides these national 
requirements, local authorities are faced with the need of 
having reliable estimates, such as demographic charac- 
teristics, for analysis, planning and administration purposes. 

In Brazil, one important example of the demand for 
reliable estimates is related to how constitutionally mandated 
federal revenue sharing is apportioned annually to the 
various municipalities (Brazil is a federated republic made 
up of states and the Federal District. The states are divided 
into municipalities, which share characteristics of cities and 
counties - they can contain more than one urban area, but 
they have a single mayor and municipal council). The 
predicted number of inhabitants in a municipality is used by 
the federal government as a criterion to distribute funding. 
Hence, there is a need to obtain reliable municipal popu- 
lation forecasts in order to fairly apply this criterion, 
regulated by federal law. 

An important source of demographic data is the annual 
Household Survey (PNAD). However, this survey is not 
designed to produce estimates at the municipal level. In 
other words, apart from a few municipalities, the municipal 
sample sizes are not large enough to yield acceptable 
standard errors when the direct survey estimates are used. 


Furthermore, a considerable number of municipalities are 
not sampled at all. 

The current approach to obtain municipal population 
estimates is based on making prediction for a larger area at 
first, and then using some auxiliary information to allocate 
the total predicted population to the municipalities. In turn, 
prediction for a larger area is done by assuming that birth, 
mortality and migration rates are the same for all 
municipalities. The major drawback of this approach is that 
it relies on the assumed model evolution. It does not take 
into account all uncertainties and does not provide, in 
general, error measures of the estimates. 

The small area estimation problem has received attention 
in the statistical literature due to the growing demand for 
detailed statistical information from the public and private 
sectors. An excellent and updated account of methods and 
applications of small area estimation can be found in Rao 
(2003). The main source of small area data is provided by 
periodic surveys whose sample sizes are not large enough to 
provide reliable estimates for the areas. One way of tackling 
this problem is to gain strength from all areas and through 
other sources of related data. As stated in Pfeffermann 
(2002), the sources of data suitable for this task can be 
classified into two categories: data obtained from other 
similar areas with respect to the characteristic of interest and 
past data obtained for the characteristic of interest and 
auxiliary information. In our demographic context, the main 
source of related data is provided by the 1991 and 2000 
censuses and a complete count of the population carried out 
in 1996. 

The aim of this work is to obtain estimates of the 
municipal populations based on survey data provided by the 
PNAD and census data. A non-structured hierarchical 
model is proposed and its fitness and predictive power are 
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evaluated. We also consider a spatially structured hier- 
archical model, in the spirit of Moura and Migon (2002), 
since the population per area and its growth pattern might be 
related to the development of its neighboring areas. For the 
sake of simplicity, from now on we respectively call the 
non-structured hierarchical and spatially structured hier- 
archical models as the Hierarchical model and Spatial 
model. 

In Section 2 the main data sources used in this work are 
described. In Section 3, the proposed models and a model 
selection criteria are presented. Applications with real and a 
simulated data are presented in Section 4. Finally, Section 5 
contains a brief summary with an outline for future research. 


2. Data set 


The input data for the models introduced in Section 3 are 
taken from the annual Household Surveys (PNADs) from 
1992 to 1999, the 1991 and 2000 census data and a complete 
enumeration of the population carried out in 1996. In order to 
evaluate the proposed approach, the municipalities of Sao 
Paulo State are considered as the areas of interest. 

In this section we present a brief description of these data 
sources, reporting their main advantages and limitations. The 
population direct estimates of sampled municipalities were 
obtained from the PNAD. As explained in Section 3, these 
estimates are regarded as the input data for making inference 
about our target parameters. The two censuses and the 1996 
population count are also utilized in our application. 

The Brazilian Demographic Census is the main source of 
information about the population. It is carried out every ten 
years, usually in the beginning of the decade. Although the 
objective is to count all the population, some enumeration 
errors are found. The magnitude of the errors is evaluated 
through a post enumeration survey carried out soon after the 
completion of the census. 

The annual Household Survey (PNAD) is designed to 
produce basic information about the socioeconomic 
situation of the country. The investigation unit is the 
household, for which yearly information about the number 
of dwellers, their gender, education level, employment, etc. 
is collected. The survey is not carried out in a census year, 
and was also not conducted in 1994 for administrative 
reasons. The sample is selected by a three-stage cluster 
sampling design. The primary and secondary units are 
respectively the municipality and enumeration areas (with 
250 households on average). The municipalities are 
stratified according to their population sizes as obtained 
from the last census. In the first stage, all municipalities 
belonging to the metropolitan regions and the state capitals 
(which in Brazil are normally the largest cities in the 
respective states) are sampled. The municipalities whose 
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populations are greater than some cutoff value are also 
included in the sample with probability one. The ones left are 
stratified and two of them are sampled from each stratum 
with probability proportional to their population sizes. 

The enumeration areas are sampled with probability 
proportional to the number of households residing in the 
area in the last census. Finally, in the last stage the house- 
holds are sampled systematically with equal probability 
from a list, which is updated at the beginning of the survey. 
The municipalities and enumeration districts are kept the 
same in all the surveys carried out in the same decade, while 
households are sampled every year. 

Since each area is sampled with probability proportional 
to its respective number of households, it could be argued 
that the sampling mechanism is informative with respect to 
the population of the area. However, since the response 
variable actually used in this work is the area density, it is 
reasonable to assume that the sample selection mechanism is 
not relevant. Thus, this issue is not exploited in this work. A 
good reference about how to make small area inference 
under informative sampling is Pfeffermann and Sverchkov 
(2007). We also recommend Pfeffermann, Moura and Silva 
(2006) for readers interested in how to employ a Bayesian 
approach to hierarchically modeling under informative 
sampling. 


3. Model specification 


3.1 Exponential growth model 


Let y, be sample values of a distribution belonging to an 
exponential family with expected value given by 1, = 
E(y,| 9,) where 9, is a vector of unknown parameters. 

An important and wide class of exponential growth 
models parameterized by (a, B, y, ») is defined as: 


m, = [a + Bexp(ys)]”. (1) 
Some special well-known cases in the literature are: 


(1) Logistic: with » = -1, n/' = a + Bexp(yt); 


t 
(2)Gompertz: with > = 0, defining (1) as log(z,) = 
a + Bexp(yt); 
(3) Modified exponential: with > = 1, 7, =a+Pexp(ys). 


The main advantage of using model (1) is the possibility 
of keeping the observations y, in the original scale, 
changing only the trajectory of m,, making interpretation 
easy. Furthermore, the time intervals do not need to be of 
the same length, allowing the data to come from different 
reference sources (see Section 4 for further details). 

When y = exp(y) <1, the process is non-explosive, 
implying that 2, converges to a’* when ¢ — oo, with the 
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convention that for @ = 0, this quantity is equal to log(a). 
When w > 1, the curves are concave for ¢>0 and 
B > 0, leading to an explosive process. This class of 
models is called the generalized exponential growth model. 
Migon and Gamerman (1993) show how the exponential 
growth model can be viewed as a particular case of a 
general dynamic model. 


3.2 Hierarchical growth models 


In this paper our main parameters of interest 7,, are 
nonlinear exponential growth functions with some parameters 
that are hierarchically or spatially structured. Spatially 
structured models provide alternative ways for connecting 
similar neighboring areas. We further assume that the 
sampling variance o; follows a model that depends on the 
sample size in the respective municipality. In this work, 
hierarchical and spatial models are fitted and compared. 

We assume that the population sizes are available for all 
the m municipalities of Sao Paulo State for the census 
years of 1991 and 2000, as well as the complete population 
count in 1996. From now on, we simply refer to them as the 
census data. In order to improve the hypothesis of 
exchangeability of the parameters describing the mean of 
the process, our response variables are set as the sampled 
municipal density estimates instead of the municipal 
population estimates. See also the end of Section 2 for 
further reasons for using the densities. 

For each period, estimates of these quantities are 
available only for k < m first-stage units municipalities of 
the PNAD sample. In order to estimate the municipal 
density, we simply divide the total population estimate by 
the respective municipal area. 

Let y, be the population density obtained from the 
census data or estimated by the PNAD at time 4, ¢ = 1, ...n 
for the i” municipality, i = 1, .... m. Our aim is to make 
inferences about the true population density 7, for the 
population of all municipalities, including those that are not 
sampled. In the next section, true municipal population 
densities 1, are modeled via a stochastic nonlinear hier- 
archical growth function. We assume that the random 
quantities y,, are normally distributed with mean 1m, and 
variance 6;,. 

We use a Bayesian approach in this work. Therefore, 
predictions are described by probability distributions, giving 
the opportunity for users to analyze the uncertainties 
involved in the decision process. This fact is one of the 
advantages, among many others, of using this kind of 
approach. 

Only in the census years are the y, obtained for all the 
municipalities of Sao Paulo State. Although the census 
attempts to obtain complete enumeration of the whole 
population, coverage errors can occur. The following model 
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is assumed therefore for the census data and the data 
obtained from the PNAD, with exception that the variances 
G;, are set to be smaller for the census data (see Section 3.4 
and also the final remarks in Section 5): 


io Sy ~ N(O, o;) 
a = {a,+ Bexp(y,f}"" 
Pe enone (0, O; ) 


Vi =¥+ bys &, ~ NO, 02) 


where the prior distributions of a, B and y are given by: 
Ga V(r, oe): B ~ N(u,, C.), ieee o,). It should 
be noted that information from all areas is obtained through 
the hierarchical structure of the parameters a,, and y,. 
Another way of borrowing information between munic- 
ipalities is to assume that a, are spatially structured (see 
Section 3.3). Supposing that the mean 1,, is non-explosive, 
the parameter a'* can be regarded as the value at which the 
mean municipal population stabilizes. The parameters B 
and y affect the evolution of the density over time. The 
prior distributions of a, B and y can be chosen by taking 
advantage of some pnor demographic knowledge of the 
expected population evolution. In our application, we set 
od = 1, implying that for t = 0 the tme value density in 
each municipality is given by a, +f. The hierarchical 
structure imposed on the parameters a,, implies that the 
expected value of the true density for any municipality at 
t= 0 is a +f. To assume that the growth parameters, y,, 
have a hierarchical structure means that the densities have 
different growth rates but share the same mean. A small 
simulation study (see Section 4.1) guides us to keep the B 
parameter fixed for all areas, without any loss of generality, 
since the levels are still different for different municipalities. 
In all models considered in our application, we assume that 
Toe Oa), p), rs =: o ~ G(a,, 6,). In order to 
assign vague priors, in Section 4.2 we set small values for 
the parameters related to these precision prior distributions. 

The assumption that the mean function 7,, 1s given by an 
exponential growth curve allows adjusting for increasing or 
decreasing population density. The sources of data used 
have different reference data and are not equally spaced in 
time. In this case, the use of an exponential growth curve 
yields an extra advantage, since we can simply make a scale 
of time in order to conform with the different data sources, 
as explained in the application section 4. 


Vi — Uy tt 


a 
| 


(2) 


3.3. Spatial model 


In the Hierarchical model presented in the previous 
section, the information from all areas is combined in order 
to predict the population of a particular area. However, it is 
reasonable to assume that two or more neighboring 
municipalities have more similar demographic densities 
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than two other arbitrarily chosen ones. The regional 
structure is represented in the joint prior distribution of the 
random spatial effects. We consider that two areas are 
neighbors if they share a border. 

In our proposed model, the demographic density in an 
area i at time /, 7,, is affected by its neighboring areas by 
adding random spatial effects 6, to the parameters a,, 
that is, a, = a+ ome where a is a term representing the 
intercept. Therefore, a, vary only with the spatial effect, 
representing a local effect, while the growth parameters 
y, s are regarded as similar among all areas (overall effect). 

The relationship between neighboring areas is defined in 
the prior distributions of 5,,. The prior joint distribution of 
Soy Ga,» s1> Sa ty given the hyperparameter o., is 
defined as in Mollié (1996): 


1 m 
Pd, | 4) 5 om”? exp a DS Wiz (5, + on Je (3) 


1 
a 2 a i=l k<i 


where w, are the weights associated with the regional 
structure. The weights were chosen such that w, = 1, if i 
and k are contiguous, and w, =0, otherwise. The 
distribution of 5,| 6% is evidently improper, since we can 
add any constant to all of the 6, and p(6, | 6.) is not 
affected. Thus, we must impose a constraint to ensure that 
the model is identifiable. We set }7,5, = 0 and assign a 
uniform prior distribution on the whole real line to the 
intercept a. It is not difficult to see that this procedure leads 
to a proper (m—1) dimensional likelihood density, see 
Besag and Kooperang (1995) for further details. 


The prior conditional distribution of 5,, given the 
effects 6, of the remaining areas and the hyperparameter 
G., is normal with mean and variance given by: 


E[6,,1 54,> & € Gi, Ci= Day 
o 
Var[5, | 5,., k € di, 02] = — 
i k Ww 


i+ 


where 5, denotes the arithmetic mean of the 5, for 
kKedi (the contiguous areas of 7), and w,, = YZ, Ww, iS 
the number of neighboring municipalities of i. 

Figure 1 shows the demographic densities of SAo Paulo 
municipalities in 1991. These municipalities tend to be 
concentrated geographically according to density classes. 
This suggests that the spatial model can be usefully applied. 


3.4 Modeling the sampling variances 


Since we use data from two different sources, it makes 
sense to assume that the sampling variances vary over time. 
Furthermore, we can also consider that the variances change 
with the areas. 

For the years in which the data are provided by the 
PNAD, we assume the following model for the sampling 
variances: 


log(o%,) =n + n,-(1/n;) (4) 


with n, representing the number of enumeration areas 
sampled in the i" area. This model captures the expectation 
that the variance gets smaller as the sample size increases. 


Density in Km? 


[__]4-69 
f= 170-184 


S54 357 
58 — 668 
@ 669 — 1,470 
1,471 — 2,478 
Be 2.479 — 6,313 
BM 6,314 — 9,968 


Figure 1 Population densities of S40 Paulo municipalities in 1991 
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For the years that the censuses were carried out, we 
assumed that 7, is known and log(o;) = log(v,) where 
v,, 18 calculated in such a way that the census coverage error 
is 5% for all areas. This hypothesis implies that the true 
population in each area for census years lies in the interval 
given by the observed population in the census plus or 
minus 5% of this value. Therefore, for the census years we 
set the standard deviation as: o,, = 0.05 * (y,,/2). Assum- 
ing known variance in the census years is a way of giving 
more weight to census data, since one would expect a 
complete census to provide more reliable information than 
survey data. Independent normal distributions are assumed 
for the parameters n, and n,: 1, ~ N(u,,, oy, ); & = 9,1. 
In order to assign vague priors to the n ’s, we set both prior 
means as zero and large values for the , ’s. See Section 4.2 
for details. 


3.5 Summary of the models 


The prior distributions of the common parameters of the 
Spatial and Hierarchical models are the same as already 
described for the former. The distributions of the random 
spatial effects are specified in Section 3.3. The variance o;, 
in the Spatial model was stated as in the Hierarchical model. 
A summary of the models in Section 4 is presented in Table 
1. For the sake of simplicity, the application was carried out 
by fixing } = 1 in both models. 


3.6 Computational issues 


The posterior distributions of the parameters for the 
models proposed cannot be obtained in closed forms. 
Therefore, it is necessary to use numerical approximation 
methods. One alternative, often used and easy to implement, 
is to generate samples of these distributions based on the 
Markov Chain Monte Carlo (MCMC) algorithm. Since the 
full conditional distributions of all the model parameters 
have closed form, except for the vector y = (¥,, ..., ¥,), we 
employed the Gibbs sampler algorithm with one acceptance/ 
rejection algorithm step for sampling from the vector y. Let 
rt, be the population density in the i" area at time ¢. The 
following steps summarize how to sample from the 
posterior distribution of 7,/: 


Table 1 
Summary of the models employed 
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1. Generate a6, B®, ¥, a, y, 22, 2 ny? and 
‘2 Pe: ’ Lider trae, 
Th” for / = 1, ..... M, where M is the number of 
MCMC samples generated from the full conditional 
distributions of all model parameters including the 
random effects; 


2. Calculate 2 = a? + BPexp(y!r); 


Three informal checks for convergence, based on 
graphical techniques, were applied for assessing the 
convergence when fitting our proposed models. They 
consist of observing the histogram, the trace and the 
autocorrelation function for each of the sampled values 
calculated. The histogram analysis allows us to identify 
possible departures from convergence, such as the presence 
of multiple modes. The trace of the multiple chains 
simulated in parallel, each one with different starting points 
and overdispersed with respect to the target distribution, 
provides a rough indication of stationary behavior when the 
sequences of values tend to oscillate in the same region. The 
plot of the autocorrelation function allows identifying 
whether the sampling can be regarded as independent. 

In addition to these informal checks, other more formal 
criteria were applied. The criteria introduced by Brooks and 
Gelman (1998) and implemented in WinBugs 1.4 
(Spiegelhalter, Thomas, Best and Lunn 2004) permit 
diagnosing whether dispersion within chains is larger than 
dispersion between chains. Consider / parallel chain and a 
parameter of interest A. Let A/ be the j™ value of the i” 
Chant Alone iliac cand aj ies, 1 hem-sthe 
variances between chains B and within chains W are 
given by 


B=J(K - D'S G, UWS 


and 
Ku) 
RE ORN) SOR CHENO 
i=] j=l 
where 4, and A respectively are the average of obser- 
vations of chain i, i = 1, ..., K and the global average. 
Under convergence, all these KJ values are drawn from 
the posterior of 1 and the variance of 4 can be consistently 
estimated by B,W and the weighted average 6; = 
(P=) WL eB, 


model parameters variance prior distribution 
Hierarchical a; = a+ Eq log(a;,) = No + n(I/n;), No ~ Muy,» On,) 
B for survey data m1 ~ My,» On, ) 
1; = eit sy oe is assumed to be 
known for census data 
Spatial (Cho = Char 8a, log(o? ) = No + M(/n;) Ope Ones 174 N(8q.5 Te/Wi+) 
B in the survey pe dq, = 0 
Yi = + &y, o7 is assumed to be No ~ N(My,> On) 


known for census data 


nies N(My,> on,) 
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If the chains have not yet converged, then initial values 
will still be influencing the trajectories and 6; will 
overestimate o; until stationarity be reached. On the other 
hand, before convergence, W will tend to underestimate 
o;. Following these reasoning, Brooks and Gelman (1998) 
proposed an iterated graphical approach, which is 
implemented in WinBugs 1.4. It allows to check if: (i) the 
weighted posterior variance estimated 6; and the within- 
chain variance W stabilize as a function of J, and (ii) the 


variance reduction factor, R = 6;/W, approaches 1. 


4. Application 


In this section we present two applications of our 
approach, the first one with a simulated data set and the 
second one with the real data set that motivated this work. 
The simulation study aims to check if the parameters of 
interest are being properly estimated, as well as to perform 
some sensitivity analysis with respect to the form of the 
prior distributions used for fitting the model. 


4.1 Application to simulated data 


We carried out a small simulation study fitting the 
Hierarchical and Spatial models presented in Section 3. The 
true model hyperparameters related to the growth curve 
were fixed as a = 40, 8 = 25, y = 0.05. Thus, we are 
considering a situation where the population size approxi- 
mately doubles in 25 years. The parameters related to the 
sampling variance model were fixed as ny = 6.5, n, =0.5. 
Finally, the precision parameters were respectively set as 
t, =0.0001 and ine = 400. The precision t2 and ce were 
fixed to be in agreement with the scales of the quantities 
they respectively measure. The intercept presents more 
relative variation between areas than the growth parameter, 
which is expected in practical situations. 

Since it is well recognized that the form of the priors has 
more impact on the component of variance parameters than 
the fixed parameters, we fitted the simulated data using two 
different vague priors for the parameters related to the 
variances: uniform for the standard deviation, which is one 
of the priors recommended by Gelman (2006) for linear hier- 
archical models, and gamma for the precision, commonly 
used as the default in some computational packages. In the 
first case, we assigned o, ~ U(0, 1,000) and ow 
U(0, 100), where o, =1/t, and o, =1/t,. In the 
second case, we considered t> ~ G(0.001, 0.001) and 
~ G(0.001, 0.001). For the other parameters, we set 
a ~ U(—«, +00), for the Spatial Model (see Section 3.3 
for further details) and a ~ N(0, 10°) for the Hierarchical 
model. For the others parameters we set B ~ N(0, 10°), 
y ~ N(O, 107), ny ~ N(O, 10*) and n,~ (0, 10*) for 
both models. The effect of the number of small areas is also 
investigated. We simulated separate data from the Hier- 
archical and Spatial models with m= 60 and m =100 
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areas in each case. For each combination of the number of 
areas and the model employed we generated 200 data sets. 
Therefore, a total of 800 sets of artificial data was simulated. 
The distribution of the sample sizes within the areas is the 
same for the simulated data sets with 60 and 100 areas. 
Table 2 presents the relative frequencies of the small areas 
sample sizes for the both simulated data sets. These sample 
sizes are very similar to the sample sizes in the real data that 
underlines this simulation study. The number of neighbors 
employed in the spatial model varies from 1 to 12 and each 
area has on average 5 neighbors. We considered a total 
period of n = 9 years. 


Table 2 
Relative frequencies of the small area samples sizes for both 
simulated data sets 


Sample size Relative frequency 
p 0.05 
5 0.20 
8 0.25 
10 0.25 
12 0.20 
15 0.05 


In order to get rid of chain correlation, we generated 
20,000 samples after discarding the first 10,000. There is no 
evidence for non-convergence of the Hierarchical and the 
Spatial model parameters. A careful analysis of some outputs 
obtained from the MCMC samples for some simulation sets 
suggests that convergence was achieved for all model 
parameters. We assessed the statistical properties of the 
population density (7,,) estimates by investigating the 
average of the absolute relative error of the estimates (ARE) 
and the mean square error (MSE), respectively given by: 


p (1) (1) 
iz l 200 |; , = 


(2 eS 1 
200 El re 


and 

l 200 » > 
yen we 
b= Am, t= 1c. Naw ere. ts- no. mica: vatiation..as 
far as the ARE values are concerned. For the two models 
fitted and both small area sample sizes tried, the ARE values 
are around 1.5%. 

Table 3 shows a summary of the MSE values obtained 
from the simulations carried out under the Spatial and 
Hierarchical models with 60 and 100 areas and respectively 
assigning gamma and uniform priors to the precision and to 
the standard deviation of the parameters related to the 
variance. It can be seen from Table 3 that the MSEs are not 
affected by the use of different vague priors. It is noteworthy 
that increasing the number of areas from 60 to 100 results in 
a small decrease of 6% in the median of the MSE for the 
Spatial model. However, for the case of the Hierarchical 
model, the decrease is about 13%. 


MSE, , = 
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Summary of mean square error distribution for the spatial and hierarchical models 


Model Num. of areas 


Spatial 60 0.398 1.741 
100 0.525 1.637 
Hierarchical 60 0.542 2.218 
100 0.594 1.959 


We also investigated the percentage coverage of nominal 
95% credible intervals. The results are presented in Table 4. 
As far as this simulation study is concerned, the intervals for 
the parameters of interest have in general the correct 
coverage percentages for both models investigated and these 
results do not depend on wether we have 60 or 100 areas. 
However, with a small number of areas we could face 
convergence problems unless we tighten the priors for the 
hyperparameters. The simulation study reveals that the 
population prediction is not affected by the forms of the 
vague priors assigned to the variance of the intercept term. 


Table 4 
The coverage rates of nominal 95% credible intervals for the 
population densities 


Model Num. of Gamma prior Uniform prior 
Areas coverage(%) coverage(%) 
Spatial 60 96 96 
100 96 96 
Hierarchical 60 94 94 


100 95 95 


We analyzed the model fit when data generated from a 
model were fitted by the correct and the wrong models. 
Figure 2 presents the mean square error for the following 
situations: (a) data generated from the Spatial model and 
fitted by the Spatial and Hierarchical models and (b) data 
generated from the Hierarchical model and fitted by the 
Spatial and Hierarchical models. Since the form of the 
priors assigned to the parameters related to the variance 
does not affect the inference, we set uniform priors for both 
models. The ARE measures are shown in Figure 3. 

It can be seen from Figure 2 that when the data are 
generated from the simpler model (Hierarchical) the more 
complex estimation procedures (Spatial) do not suffer any 
appreciable worsening of efficiency. On the other hand 
when the data are generated from the more complex model 
(Spatial) the simpler estimator (Hierarchical) has some 
inferior properties. However, this result does not hold for the 
ARE measurements. Figure 3 shows that fitting the model 
not used for generating the data results in appreciable 
increase in the relative bias. As it might be expected, model 
fitting and diagnostics are crucial in order to get suitable 
prediction of the small area population. 


Gamma prior 


Uniform prior 
Median 


3° Qu. u. 
0.394 


3.538 0.524 1.641 3.517 
6.262 0.646 Des 6.278 
9593 0.596 1.960 5.619 


4.2 Application to real data 


The PNAD data sets from 1992 to 1999, (excluding 1994 
and 1996) and the population census data of 1991, 1996 and 
2001 were used in our application. Our areas of interest are 
all the municipalities in SAo Paulo State, a total of 572 areas, 
of which 111 areas were sampled by the PNAD survey. 
Figure 4 shows the areas sampled by the PNAD, classified 
by the sampling definition: areas belong to metropolitan 
regions and self-representing areas (sampled with probability 
equal to 1) and non-self-representing areas. It should be 
noted that the census and PNAD have different periods of 
reference. We set ¢ = 0 for the 1991 census. Thus, the 
values of ¢ for the data provided by the PNAD are equal to 
the number of years between the reference period of the 
1991 census and the respective PNAD. For instance, a 
survey datum provided by the PNAD 18 months after the 
1991 census corresponds to ¢ = 1.5. 

Figure 5 shows the estimated coefficient of variation of 
the direct estimator by areas’sample sizes. These estimates 
are based on PNAD data. It can be seen that these 
coefficients of variation vary considerably with the areas 
and tend to decrease as the sample size increases. The high 
values of these coefficients show the difficulty in using only 
the direct estimator to provide municipal estimates. Further- 
more, we cannot make any prediction for nonsampled areas 
by using only the direct estimators. 


4.3 Specification of the prior distributions 


The mean of the normal pnor distributions of the 
parameters a, 8 and y, related to the population evolution, 
were assigned by first expanding the function a + 
Bexp(yt) around zero in a Taylor series up to the second 
order and then equating the resulting expression to the 
values of the mean density in the 1991 and 2000 censuses 
and the 1996 population count. In the absence of prior 
information, we considered a reasonably large value (10°) 
for the prior variances of a, B and y. Thus, we set a ~ 
U(-o0, +00) (see Section 3.3 for further details), for the 
Spatial Model and a ~ N(370, 10°), for the Hierarchical 
model and B ~ N(726, 10°), y ~ N(0.04, 10°) for both 
models. The reason for this adjustment is to obtain a reason- 
able value of the prior means, but one that is essentially 
vague. Regarding the precisions and 1, n,, we assigned 
relatively vague priors: Ka ~ Ga(0.001, 0.001), By ~ 
Ga(0.001, 0.001), n)~ N(0, 10°) and n, ~ N(0, 10°). 
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Spatial Model Hierarchical Model Spatial Model Hierarchical Model 
(a) (b) 

Figure 2 Box plots of mean square error (MSE) for the cases: (a) data generated 
from the Spatial model and respectively fitted by the Spatial and 
Hierarchical models and (b) data generated from the Hierarchical model 
and respectively fitted by the Spatial and Hierarchical models 


Spatial Model Hierarchical Model Spatial Model Hierarchical Model 
(a) (b) 


Figure3 Box plots of absolute relative error (ARE) for the cases: (a) data 
generated from the Spatial model and respectively fitted by the Spatial 
and Hierarchical models and (b) data generated from the Hierarchical 
model and respectively fitted by the Spatial and Hierarchical models 
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Figure 4 Sao Paulo municipalities sampled by the PNAD classified by the sampling definition 
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Figure 5 Boxplot of the coefficients of variation of the direct population estimates 


4.4 Some results 


We generated 20,000 samples after discarding the first 
5,000. There is no evidence for non-convergence of the 
Hierarchical and the Spatial model parameters. A careful 
analysis of the MCMC outputs suggests that convergence 
was achieved for all model parameters. We summarize the 
results obtained by fitting the Hierarchical model (3) to the 
data provided by the PNAD survey. The posterior means of 
the model parameters were used as the point estimates. 
Table 5 presents these estimates together with the respective 
square root of the posterior variance. It can be seen from 
Table 5 that the estimate of n, 1s significantly positive, 
which agrees with what is expected by equation 4: the 
greater the sample size, the smaller o%. 


Table 5 
Summary of the model (2) parameter posterior distributions 


parameter posterior mean posterior std 
a 892.500 202.000 
B 105.700 1.278 
Y 0.072 0.008 
No 10.620 0.133 
an 3.185 0.484 
ms 2.174E-7 2.961E-8 
ts 139.000 19.560 


Figure 6 shows that the posterior means of the 
parameters a and y that index the hierarchical model seem 
to be spatially distributed. The parameters of neighboring 
areas seem more alike than those of distant areas, which 
suggests applying the Spatial model. 


4.5 Model selection 


The Expected Prediction Deviance (EPD) (Gelfand and 
Ghosh 1998) measure was applied to help choose the most 
suitable model. The EPD measure is the sum of two terms. 
The first term, denoted by G, can be interpreted as a 
goodness-of-fit measure and the second term, denoted by P, 
as a penalty term for underfitted as well as overfitted models. 
The respective expressions for G and P are given by: 
G=TET(%,-EO |My? and P=D2,57,V(y'|M), 
where the expectations and the variances are with respect to 
the posterior predictive distribution associated with a future 
observation (y,;°) of y, generated under the assumed 
model (M). According to this criterion, the smaller its value, 
the better the model. As can be seen in Table 6, the EPD 
criterion slightly favors the Spatial model. 


4.6 Analysis of the results 


The most disaggregated level for which the PNAD 
provides precise estimates is the metropolitan region, which 
is a set of contiguous municipalities. In order to validate the 
results obtained with the spatial model, population estimates 
for the greater Sao Paulo metropolitan region were 
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compared to the official statistics projections. The posterior 
distribution of wu, = >7_,7,, * A, 1s easily obtained by 
adding 10? = 7,2‘? * 4, to the MCMC algorithm, where 
Lt, represents the total population of the metropolitan region 
at time ¢ and r is the number of municipalities belonging 


to that metropolitan region. 


Table 6 

Measures for selecting models for demographic density 
Model G P EPD 
Hierarchical 1.372409 6.14E+09 T.SLE+09 
Spatial 1.0S5E+09 6.19E+09 7.24E+09 


Figure 7 compares the population estimates (u,) of the 
Sao Paulo metropolitan region obtained by the Spatial 
model and the official statistics. The solid lines represent the 
limits of the 95% credible intervals of p1,, while the dotted 
line shows the respective point estimates. The symbol (+) 
represents the observed official statistics. It is noteworthy 
that some official statistics projections are outside of the 
credibles inferior limit (including the 1991 Census). This 
indicates that further investigations should be made in order 
to find out the reasons for these discrepancies. However, 
when we compare them at municipality level, the overall 
conclusion is that the model predictions and official statistics 
reasonable agree. The 95% credible intervals contains 92.4% 
of the official statistics projections. The average of the 
absolute relative error (ARE) between the estimated popu- 
lation density and the official statistics projection are 3%. 
These ARE measures are on average nearly the same for 
selected and non-selected municipalities. 

Figure 8 compares the point estimates of the population 
sizes (u,,) With the official projection statistics and the 
official census population sizes for a sampled municipality. 
The official projection methodology assumes that a set of 
small areas and a larger area, which contains them, have the 
same population growth rate pattern. The population of the 
larger area is projected by a component method and then 
proportionally allocated to the small areas. The component 
method uses data from the most recent census as well as the 
number of births and deaths and net migrations obtained 
from administrative records. The component method 
projects the population for a time ¢ by adding the 
population in a previous time with the number of births and 
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net migrations and subtracting the number of deaths in the 
same time interval. 

The solid lines represent the 95% credible intervals for 
u,, obtained by the Spatial model, while the dotted line 
shows the respective posterior means. The symbol (+) 
represents the official population projection for the inter- 
census period and the observed population in the census 
years. It is noteworthy that the point estimates are relatively 
close to the official projection statistics and the population 
obtained in the census year. This indicates that the use of the 
proposed model yields reliable estimates at municipality 
levels, with the extra advantage of providing a measure of 
the respective error. 

We also analyze the estimates obtained for some 
municipalities not sampled in the PNAD. Figure 9 shows 
the model predictions, the 95% credible intervals, the 
official projection statistics and the observed population 
values in the censuses for a non-sampled municipality (+). It 
can be seen that the predictions obtained by the Spatial 
model reasonably agree with the official figures. 


5. Final remarks 


The model used in this article identifies the population 
growth trend of the municipalities. Reasonable estimates of 
the municipal populations are obtained for years with survey 
data, as well as for the years where census data are 
available. The point estimates have good precision and 
reasonably agree with estimates obtained for larger areas 
using other technique. The past information can be updated 
as soon as estimates become available from a new census or 
survey. Furthermore, the proposed approach provides the 
probability distribution of the quantity of interest, aiding the 
decision-making process. 

Further work should be done in order to allow for 
autocorrelation of the parameters of interest over time. Extra 
information about the sampling variance estimates of the 
direct estimators could also be regarded as additional data. 
The assumption that the census coverage error is distributed 
symmetrically around zero could be relaxed by assigning a 
non-symmetric distribution to it. However a _ good 
knowledge of the shape of the distribution is required, 
which might be difficult in practice. 
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Figure 6 Posterior means of the parameters a and y obtained by the hierarchical model 
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Figure 7 Comparison between the population sizes predicted by the spatial model and the 
official statistics (+) for the metropolitan region 
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Figure 8 Comparison between the population sizes predicted by the spatial model and the 
official statistics (+) for a sampled municipality 
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Figure 9 Population sizes predicted by the spatial model and the official statistics (+) for a 


non-sampled municipality 
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Variance estimation in the presence of nonrespondents 
and certainty strata 


Jun Shao and Katherine J. Thompson ' 


Abstract 


Business surveys often use a one-stage stratified simple random sampling without replacement design with some certainty 
strata. Although weight adjustment is typically applied for unit nonresponse, the variability due to nonresponse may be 
omitted in practice when estimating variances. This is problematic especially when there are certainty strata. We derive 
some variance estimators that are consistent when the number of sampled units in each weighting cell is large, using the 
jackknife, linearization, and modified jackknife methods. The derived variance estimators are first applied to empirical data 
from the Annual Capital Expenditures Survey conducted by the U.S. Census Bureau and are then examined in a simulation 


study. 


Key Words: Covariate dependent nonresponse; Jackknife; Linearization; Ratio adjustment; Uniform nonresponse. 


1. Introduction 


Many business surveys use a one-stage stratified simple 
random sample without replacement design. Because of the 
skewness of the sampled populations, these designs 
generally include both certainty and non-certainty strata. 
With such designs, the sampling rates in the non-certainty 
strata are generally negligible (e.g., less than 20 percent in 
all strata). However, if the ultimate sampling unit is large 
business entity such as a company, the size of the universe is 
much smaller and often sampling fractions should not be 
ignored in computation of variance estimates. 

Most surveys have nonresponse. We consider surveys 
using weighting adjustment for nonresponse. For certainty 
strata, there is no sampling error and, hence, standard 
variance formulas do not include any component for certain- 
ty strata. When nonresponse is present, however, there is an 
estimation error even in a certainty stratum, which is often 
an appreciable component of the total estimation error. 

The purpose of this paper is to develop some methods for 
variance estimation that take into account the weighting 
adjustment for nonresponse and the existence of certainty 
strata. After introducing notation and assumptions in 
Section 2, we show that the jackknife and linearization 
variance estimators ignoring nonresponse in certainty strata, 
which are often currently used in many surveys, under- 
estimate the true variance of the weight adjusted estimated 
population total. By directly deriving an approximate 
variance formula, we obtain two consistent variance esti- 
mators. These variance estimators are also consistent if there 
are non-certainty strata with large sampling fractions. A 
modified jackknife variance estimator taking into account 
the variability due to nonresponse in certainty strata is also 
derived. 


In Section 3, we compare variance estimators using five 
years’ of data from the Annual Capital Expenditures Survey 
(ACES) conducted by the U.S. Census Bureau. Simulation 
results are presented in Section 4 using a population 
generated from 2003 ACES data. Our simulation results 
show that the variance estimators ignoring certainty strata 
have large negative biases; the derived consistent variance 
estimators perform well when stratum sample sizes are all 
large and perform inconsistently otherwise; and _ the 
jackknife variance estimator ignoring all sampling fractions 
overestimates. Some concluding remarks are given in 
Section 5. 


2. Main results 


Consider a stratified sample without replacement from a 
finite population containing H strata. Let n, and N, be 
the sample and population size of stratum h, respectively, 
y,; be a variable of interest that may have nonresponse, and 
x,; be a covariate that takes positive values and does not 
have nonresponse, where j is the index of population unit 
and /h is the index for stratum. Using the sample-response 
path considered by Fay (1991) and Shao and Steel (1999), 
we view the finite population as a census with y, x values 
and nonrespondents, i.e., each unit j in stratum / of the 
finite population is associated with an indicator /,,(=1 if 
y,; 18 arespondent and = 0 if y,, is a nonrespondent). Our 
sample is taken from this finite population, and if unit 7 in 
stratum / is in the sample, y,, is a respondent if /,, = 1 
and a nonrespondent if /,,, = 0. 

Let E, and V, be the expectation and variance, 
respectively, with respect to sampling and E,V,, and P,, 
be the expectation, variance, and probability, respectively, 
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with respect to the model m specified in one of the 
following assumptions. 


Assumption M. Values of (y,;, x,;, Z,,) in the finite 
population are independently generated from a 
superpopulation model m. The finite population is 
divided into P sub-populations such that, within sub- 
population p, the response probability P,(/,, = 
L|Yyjs%y) =F, Uy = 1 Zw) > 0, E,, (Vij | Xn) = Bp yj» 
and V,.(¥x; | x4) = o p*» Where B, and o, are 
unknown parameters depending on p. 


Assumption P. The finite population is divided into P 
sub-populations such that, under a superpopulation 
model, P,(,,=1| Yj. %;) =%,>9 is constant 
within sub-population p. 


The sub-population in Assumption M or Assumption P is 
called nonresponse adjustment weighting cell (or weighting 
cell for short), since we handle nonrespondents by weight 
adjustment within each weighting cell. (If imputation is 
applied within each sub-population, then sub-populations 
are called imputation cells.) In applications, weighting cells 
may be strata, or unions of strata (strata are collapsed when 
they have insufficient respondents), or may cut across strata. 
Assumption M involves a prediction model between y,, 
and x, and a covariate-dependent response mechanism 
within each weighting cell. The response mechanism under 
Assumption P is the within-weighting-cell uniform response 
mechanism and is often referred to as the quasi-random 
response model. Assumption P is stronger than Assumption 
M in terms of the response mechanism. However, 
Assumption M requires an explicit model between y,, and 
x,; Within each weighting cell. In this paper we assume 
either Assumption M or Assumption P. Estimators that can 
be justified under Assumption P are referred to as the 
“quasi-randomization” estimators (Oh and Scheuren 1983). 

When we study asymptotic consistency of estimators, we 
consider the limiting process of k, — oo for all p with 
fixed H and P, where k, is the sample size in weighting 
cell p. If weighting cells are the same as strata or unions of 
strata, then k,, — oo is the same as n, — © forall A. 

After the ratio-adjustment for nonresponse, we consider 
~ the following estimator of the total of y -values in the finite 
population: 


“SEE [Fem |P lan DZ 


Pp A 
A pr? 
h jes, P gr 


where p is the index for weighting cell, s, is the sample in 
stratum A, 5,,, is the indicator for the weighting cell p, 
and w,, is the survey weight constructed for the stratified 


sampling, 
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JES), 


and 


In the special case where weighting cells are the same as 
strata, 


Yay (2) 


where 
(oii yy Wij Xnp> =) Wry Xl ys 
Jesh JES, 
and 
Sia ” Way VajLiy- 
JES), 

When the covariate x,, = 1, Y is referred to as the count 
estimator. The count estimator controls respondent estimates 
to frame population totals. When the weighting cells are the 
same as strata, the count estimator uses the unweighted cell 
response rates, as recommended in Vartivarian and Little 
(2002). 

Under Assumption M or P, 


E_E(Y -Y) = E,E,(Y —Y)=0 


where Y is the finite population total of y values, and the 
total variance 


V,, (Y - Y) = E,,[V,(Y)] + V,,[E,(Y) - Y]. 


Let V, = E,[V,(Y)] and V, = V,[E,(Y) — Y]. To estimate 
V,, it suffices to estimate the sampling variance V(Y ). 
Since Y defined by (1) is a sum of ratios and each of X - 
X,,, and Y,, is a weighted total of variables and 
indicators, we can apply the stratified jackknife variance 
estimator 


n n, —1 
Vv, = }-—+ | 
J\ | a n, 5 [2 (hj) 


JESp, 


a 
—y a (3) 
Ny kes, 
(see Wolter 1985 or Shao and Tu 1995), where 6 (aj) 1S the 
jackknife analog of Y when unit j in stratum h is deleted. 
Note that sampling fractions are incorporated in this 
formula. When &, + © for all weighting cells, the 
standard result for the complete data case (see, e.g., Krewski 
and Rao 1981) implies that the jackknife estimator v,, is 
consistent for the sampling variance V(Y ), under 
Assumption M or P. Since V, is the expectation of V,(Y), 
v,, 18 also consistent for V, under some minor conditions. 
Since the function in (1) is the sum of ratios and data in 
different weighting cells are independent, a linearization 
estimator of V, (Y) can be derived using Taylor’s expansion. 
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When weighting cells are the same as strata, for example, Y 
is given by (2) and is a separate ratio estimator whose 
linearization variance estimator can be obtained using 
standard techniques. An alternative way to derive a lin- 
earization variance estimator is to linearize the jackknife 
estimator v,, (Thompson and Yung 2006). The resulting 
estimator is 


n Ke 

ide ws % Pas De se (Son — Wij prj! prj) 
h Ny, l jes, oe 

F 2 

Le 

f. x, ACH | , (4) 


where €¢,, Bie ve DOES SC oh =, Lies, My pn dn pry 
and X,, = Lae DRS ee The estimator in (4) is 
exactly. ate same as the standard linearization variance 
estimator for the separate ratio estimator in (2) when 
weighting cells are the same as strata. Like v,,, v,, is 
consistent for V, when k, — o under Assumption M or 
P, which follows from the standard result for the complete 
data case (Krewski and Rao 1981). 

Since ratio is a smooth function, under Assumption M 
or P, 


: ay E(X. VERY DarY: 
EY) =>. E,| 4 Ss races aed PEs ey, Parry 
P 2G P EX.) 12 1 
where 
x, = y 9 ony Xhp> 
h jF% 
= ae 9 prj! nj Xhj> 
h j&F%, 
=> Ne S onjd ny Yay 
h jE% 


and 7; is the finite population in stratum h. Let Y, be the 
same as X,, with x, replaced by y,,. Then 


> V,,| HS ~y, |. 
je 


V, =V,[E,) - Y) = 


pr 


Note that V, is small if the nonresponse rate is low (V, = 0 
if there is no nonresponse) or if the model under 
Assumption M is highly predictive. If the overall sampling 
fraction, >,n,/d,N,, converges to 0, then V,/V, 
converges to 0 and, hence v,, and v,, are consistent 
estimators of the total variance Zee aaah taal maha 
Note that V, does not contain the variation from certainty 
strata due to nonresponse. Because the y-values from 
certainty strata are influential in the total Y in many 
surveys, and because in applications it is difficult to tell how 
small > ,n,/>),N,, has to be for the convergence V,/V, +0 
to take place, it is necessary to estimate V,. 
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Under Assumption M, let jo Ve, and ce be the condi- 


tional expectation, variance, and covariance, respectively, 
given all x -values and response indicators. Since 
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Under Assumption P, let V be the variance with respect to 


I, °s. Since 
Me 
I po pr she 
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pr 
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Pp pr pC I Pp’ pr 
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Since X G and X pr can be estimated by bg , and X " 


respectively, to estimate V, we only need to find an esti- 
mator of o% or Se >. Under Assumption M, a regression 
estimator of B,, is Y, aA Gi and a consistent estimator of 
oe based on regression Reis is 

A 2 

xu 
j V hj = =, 
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phj~ hj 
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pr h jes, 
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From the theory of sampling, GF is also a consistent 
estimator of S under Assumption P. Hence, under 
Assumption M or P, a consistent estimator of V, is 


ayy) & 
Viet Die + " i, (5) 


pr 


The subscript L indicates that this estimator is based on 
linearization. 

In some applications },n,/>,N, 1s negligible and non- 
response in noncertainty strata has negligible contribution to 
the variance component V,, i.e., 


Lis i & 


V,=V, ; (6) 


m| 


P 


cpr 
where the subscript c stands for certainty strata, 


Ge =), » 8 aj Xaj> =» ys 8 onjd nj Xny 


hel jé%, hel jéE#, 
ee = Ds y 9 say Yaj> eS = ys » Bonita Yaj> 
hel jeF, hel jéEF, 


and ¢’ is the collection of indices of certainty strata. A 
consistent jackknife estimator of V, can be obtained as 
follows. Note that X,,, X,,,, and Y,,, are estimators, since 
F, = 5, for h eC, but ¥,, is not an estimator because of 
nonresponse. Thus, we cannot apply the jackknife to the 
function X’,,Y.,,/X,,— Y.,. From the previous derivation 
we note that, under Assumption M, 


Ar 
oa ze - 1, 


18 cpr 
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Similarly, under Assumption P, the result holds with V,, 
replaced by V.’. Hence, we can apply the jackknife to the 
estimator X.,Y.,,/ Xp,- Let 


erie: 


and Yinjy be the jackknife analog of Y after unit j in 


heC is deleted, when we treat X, pM epr! X cpr as esti- 


mators. Then a jackknife estimator of V, is 


N, -1 & 
V2 =), = 3 [rw -7 Says Fo} 
heC h J&F, N,, keP, 
(n, =N, and s,=Z when heC). The factor 
1-X Oe in the formula for Y makes the 


cpr 
appropriate Rhus for nonresponse. Under Assumption 
Dee) egies is the response rate, which can be view 
as a “sampling” fraction for certainty strata. 


Statistics Canada, Catalogue No. 12-001-X 


Shao and Thompson: Variance estimation in the presence of nonrespondents and certainty strata 


The resulting jackknife estimator of the total variance 
Vol no tdsmethen re Vigetaiyg- « SINCE, iy sige Ges, 
1-n,/N, = 0) if stratum A is a certainty stratum, it is 
easy to see that v,, + v,, 1s equal to 


n, —1 
va a 5 [fo -2 


a rw) (7) 


NM, jes, Nh kes, 
where 
Y (nj) if stratum h is a 
, certainty stratum 
Yn) 
, 1th if stratum / is not a 
Sia N, certainty stratum. 


Compared with the jackknife variance estimator v,, in (3), 
v, in (7) addresses the variability due to nonresponse in 
certainty strata, whereas v,, does not. Under (6) and 
Assumption M or P, v, is consistent. 

Finally, the jackknife estimator that ignores all sampling 
fractions is: 


=] 


nN, 


E (fay 


JES), 


= hw (8) 


Ny, kes, 


This estimator seems to be conservative, although it is not 
theoretically justified. 
In summary, we have the following estimators of the 
total variance VEE): 


1. The jackknife estimator v,, defined in (3), which 
underestimates when V,/V, is not negligible. 

2. The linearization estimator v,, defined in (4), 
which is asymptotically equivalent to v,,. 

3. Vv, =V,, + ¥,, with v,, is defined in (5), which is 
consistent. 

4. Vy, = Vj, + Vz, Which is asymptotically equivalent 
to v,. 

5. The jackknife variance estimator v, defined in (7), 
which is consistent when (6) holds. 

6. The jackknife estimator 1. 


Under stratified simple random sampling and Assump- 
tion P, v, is approximately the same as the variance 
estimator obtained by treating the set of respondents as an 
additional phase of the stratified simple random sample (i.e., 
a two-phase sample design) and applying standard variance 
formula (when x, =1) or the variance formula for 
calibration estimators (Kott 1994, Sarmdal, Swensson and 
Wretman 1992, and Hidiroglou and Samdal 1998). This 
variance estimator, however, is not consistent when 
Assumption P does not hold. 
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3. Empirical comparisons 


In this section, we apply the variance estimators 
described in Section 2 to five years of empirical data from 
the employer component of the ACES introduced in Section 
1. Section 3.1 provides background on the ACES analysis 
variables, sample design, and estimation procedures. Section 
3.2 presents the empirical comparisons. 


3.1 Background of ACES 


The ACES collects data about the nature and level of 
capital expenditures in non-farm businesses operating 
within the United States. Respondents report capital 
expenditures, broken down by type (expenditures on 
Structures and expenditures on Equipment) for the calendar 
year in all subsidiaries and divisions for all operations 
within the United States. 

The ACES universe contains two sub-populations: 
employer companies (ACE-1) and non-employer (ACE-2) 
companies. (A nonemployer company is one that has no 
paid employees, has annual business receipts of $1,000 or 
more ($1 or more in the construction industries), and is 
subject to federal income taxes. Most nonemployers are 
self-employed individuals operating very small un- 
incorporated businesses, which may or may not be the 
owner’s principal source of income). Different forms are 
mailed to sample units depending on whether they are ACE- 
1 companies or ACE-2 companies. New ACE-1 and ACE-2 
samples are selected each year, both with stratified simple 
random sample without replacement designs. The ACE-1 
sample comprises approximately seventy-five percent of the 
ACES sample (roughly 46,000 companies selected per year 
for ACE-1, and 15,000 selected per year for ACE-2). In the 
ACE-1 design, units are stratified into size-class strata 
within each industry on the sampling frame. There are five 
separate ACE-1 strata in each industry, consisting of one 
certainty stratum (referred to as stratum 10) and four non- 
certainty strata defined by company size within industry 
(denoted by 2A through 2D, ranked from largest to smallest 
within industry), with approximately 500 non-certainty 
strata in each year’s design. Sampling fractions in the 
large-size class-within-industry strata (2A) can be fairly 
high: in most years, approximately 55% of the sample in 2A 
strata are sampled at rates between 0.5 and 1. Sampling 
fractions in the other three size class within-industry strata 
are usually less than 0.20. Design weights range from 1 to 
1,000, depending on industry and size-class strata. The 
ACE-2 component is much less highly stratified, with 
between a total of six to eight size-class strata used each 
year, and sampling fractions less than 0.01 in all strata. Our 
empirical analysis is restricted to the ACE-1 component of 
the survey, which meets all of the conditions described in 
the previous section. 
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The ACES publishes total and year-to-year change 
estimates. Estimates are published for the entire survey, and 
by industry code as indicated by the respondent units (not 
necessarily the industry code on the sampling frame). If 
there is no nonresponse, variances are estimated using the 
delete-a-group jackknife variance estimator (Kott 2001). To 
account for unit nonresponse, the ACE-1 component uses 
the ratio-adjustment procedure presented in Section 2 with 
administrative payroll data as the auxiliary variable x. 
Weighting cells are the design strata, provided that there is 
at least one respondent in the cell. Cell collapsing is 
extremely rare and is hereafter ignored in this paper. More 
details concerning the ACES survey design, methodology, 
and data limitations are available on-line at http://www. 
census.gov/csd/ace. 

Although the ACE-1 survey design is fairly typical for a 
business survey, the collected data are not. Smaller 
companies often report legitimate values of zero for capital 
expenditures, and consequently the majority of the estimates 
are often obtained from the certainty and large non-certainty 
(2A) companies. As the capital expenditures are further 
cross-classified, the incidence of reported zeros (especially 
among smaller companies) increases. 


3.2 Comparisons 


To assess the effect of the unit non-response weight 
adjustment procedure on the ACE-1 standard errors, we 
computed variance estimates from unit nonresponse 
adjusted ACE-1 data using the ratio estimator with payroll 
as the auxiliary variable, in four industries, each with high 
sampling rates in the large company non-certainty strata 
(2A). The selected industries represent a cross-section of the 
sectors represented in the ACES. These industries and their 
North American Industrial Classification System (NAICS) 
codes are: Oil and Gas Extraction (211100), Nonmetallic 
Mineral Mining and Quarrying (212300), Other Miscel- 
laneous Manufacturing (339900), and Architectural, Engi- 
neering, and Related Services (541300). In subsequent 
tables and discussions, industries are referred to by their 
NAICS code. 

Table 1 presents variance estimate comparisons using 
five years’ of ACE-1 survey data for three characteristics: 
the total capital expenditures (Total), capital expenditures on 
structures (Structures), and capital expenditures on equip- 
ment (Equipment). For comparison, the variance estimates 
are presented as a ratio to v,, in Table 1. The estimated 
totals are also included. (Note that these totals are not the 
same as the published estimates, since they are computed 
using the industry classification on the frame, not the 
industry classification provided by the respondent). 

As expected, the jackknife estimator v,, and the 
linearization jackknife estimator v,, are very close for all 


Statistics Canada, Catalogue No. 12-001-X 


220 Shao and Thompson: Variance estimation in the presence of nonrespondents and certainty strata 


variables. The consistent variance estimators (v, and v,, ) 
are all noticeably larger than their corresponding jackknife 
counterparts (v,, and v,,). In general, most capital 
expenditures are reported by certainty or large non-certainty 
companies, so effect on variance estimation of including 
non-respondent component in the variance estimator is 
noticeable. The jackknife estimator v,, which adjusts for 
the effect of certainty strata, is generally between v,, and 
v,,. In some cases, v, is equal to or very close to v,,, 
indicating that the variability due to nonresponse mainly 
comes from non-certainty strata with large sampling 
fractions. The jackknife estimate 1,, which ignores 
sampling fractions, is much larger than any other estimates. 


4. Simulation results 


In this section, we present a simulation study using data 
modeled from the ACE-1 industnes presented in the 
previous section. Section 4.1 describes the simulation 
settings. Section 4.2 presents and summarizes the results. 


4.1 Simulation settings 


We modeled our population using respondent data from 
the 2003 data collection of the three key items collected by 
the survey (Total, Structures, and Equipment). Frame data 
for the auxiliary variable (payroll) were available for all 
units. The complete population data were generated using 
the SIMDAT algorithm (Thompson 2000) with modeling 
cells equal to sampling strata and population size equal to 
the original frame size in each cell. Table 2 provides sam- 
pling fractions and correlation coefficients with the payroll 
for the modeled data in each stratum. 

In the simulation, stratified simple random samples were 
selected from the generated population. We examine the 
statistical properties of the six variance estimators described 
in Section 2 over repeated samples under the following two 
different response mechanisms applied to the sample data: 

1. The covariate-dependent response mechanism ob- 
tained by randomly applying response propensities 
modeled from the survey data with payroll as the 
covariate, which yields very high probabilities of 
responding to the large units and very small 
probabilities to the small (non-certainty) units; 

2. The within-stratum uniform response mechanism 
obtained by using the observed survey response rate 
as the within-stratum response probability. 


On the average, response probabilities in the individual 
stratum within industry were 0.85, 0.76. 0.77, 0.76, and 0.68 
for strata 10, 2A, 2B, 2C, and 2D, respectively. 

We selected 5,000 samples from the population, 
computed Y in (1) from each sample with nonresponse and 
weight adjustment, and computed the empirical mean and 
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variance of the 5,000 Y values. This was done for each 
industry and each item, with two adjustment methods: the 
ratio and count estimators. When Y is the ratio estimator 
using the payroll as the auxiliary variable, the absolute value 
of the empirical relative bias is under 1.4% and is smaller 
than 1% in most cases. For the count estimator under the 
within-stratum uniform response mechanism, its absolute 
value of the empirical relative bias is under 0.5%. The count 
estimator is not approximately unbiased in theory under the 
covariate-dependent response mechanism. In the simulation, 
however, its absolute value of the empirical relative bias is 
under 1% in most cases and has a maximum value of 2.7%. 
The empirical variance of the 5,000 Y values was used as 
the “true value” of the variance of Y. 


4.2 Results 


In 2,000 of the 5,000 samples, we computed the six 
different variance estimates for all three items, four in- 
dustries, and two weight adjustment methods. We examined 
the statistical properties of each of variance estimator over 
repeated samples using the relative bias (RB) defined as 


the average of 2,000 variance estimates 


b) 


the true variance 
the stability (ST) defined as 


,/the empirical mean squared error of variance estimate 


the true variance 


b) 


and the error rate (ER) defined as the empirical propor- 
tion of the approximate 90% confidence intervals (Y + 
1.645./ variance estimate ) from 2,000 samples that do 
not contain the true population total. 

Tables 3 and 4 respectively report the simulation results 
under the two response mechanisms. The results from these 
tables can be summarized as follows. 


1. Two variance estimators ignoring V,,v,, and v,,, 
have large negative relative biases in general. The 
error rates of the related confidence intervals are 
also large. 

2. Two consistent variance estimators, v, and v,,, 
have very similar performances and are generally 
much better than v,, and v,, in terms of the 
relative bias and the error rate of the related 
confidence intervals. 

3. The jackknife variance estimator v, performs well 
in industries 339900 and 541300, but may have 
large positive relative biases in industries 211000 
and 212300. We think that this is a “small sample” 
effect, since v, is justified by asymptotic 
consistency and the sizes of the certainty strata in 
industries 211000 and 212300 are 26 and 30, 
respectively (Table 2). The sizes of the certainty 
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strata for the other two industries are 158 and 160, 
respectively. In fact, the performance of v, and v,, 
is generally better in industries 339900 and 541300. 

4. In some cases v, has more than 10% negative 
relative biases, which is caused by the fact that some 
non-certainty strata have large sampling fractions, 
i.e., the approximation (6) does not hold enough. 

5. The jackknife variance estimator *, ignoring all 
sampling fractions has very large positive relative 
biases and is too conservative. 


5. Concluding remarks 


When nonresponse is present in certainty strata (or strata 
with large sampling fractions), the jackknife and the 
linearization variance estimators that ignore certainty strata 
(or strata with large sampling fractions) are not acceptable 
because of their large negative biases. We derive two 
asymptotically unbiased and consistent variance estimators 


Table 1 


wed 


by adding an extra term that accounts the variability from 
nonresponse in certainty strata (or strata with large sampling 
fractions). We also derive a modified jackknife estimator 
that is consistent when the certainty strata are the only strata 
that contribute to the variance due to nonresponse (i.e., 
Assumption (6) holds). 

Our simulation results show that the three derived 
variance estimators perform well when stratum sample sizes 
are all large and perform inconsistently otherwise, and that 
the jackknife variance estimator that ignores all sampling 
fractions is very conservative. 

Compared with the linearization method, the jackknife 
requires more computational resources but it has other 
advantages such as being easy to program, using a single 
recipe for different problems, and not requiring complicated 
or separate derivations for different estimators. Our 
linearization variance estimator given in (4) is in fact 
obtained by linearizing the jackknife estimator in (3). 


Variance estimates for Y with ratio adjustment in ACE-1 survey 


A 


Industry Item Year Y 

211000 Total 2002 1.63E+7 
2003 2.28E+7 

2004 2.30E+7 

2005 3.08E+7 

2006 4.18E+7 

Structures 2002 eS iE, 
2003 1.86E+7 

2004 1.70E+7 

2005 2.64E+7 

2006 SYS Sat 

Equipment 2002 3.20E+6 
2003 4.18E+6 

2004 6.01E+6 

2005 4.33E+6 

2006 6.31E+6 

212300 Total 2002 1.56E+6 
2003 1.33E+6 

2004 2.01E+6 

2005 1.96E+6 

2006 2.28E+6 

Structures 2002 D22ETS 
2003 1.49E+5 

2004 4.14E+5 

2005 BINS 

2006 2.20E+5 

Equipment 2002 1.33E+6 
2003 1.18E+6 

2004 1.60E+6 

2005 1.73E+6 

2006 2.06E+6 


Yu id) bye Male al VI 
YI 

Laat Yi YI Leal Yat 

4.63E+11 0.97 1.14 ei 1.00 Wes 
6.87E+12 0.95 121 1.26 1.00 2.81 
2.45E+12 0.98 1223 1S 1.00 4.77 
4.29E+12 0.98 122) 1.24 1.19 4.77 
6.29E+12 0.99 No lla/ 1.19 1.00 8.78 
3.99E+11 0.97 1.14 eal 1.00 15.3 
5.78E+12 0.94 eo | 27) 1.00 2.78 
8.39E+11 0.99 1.42 1.43 1.00 IGS 
3.84E+12 0.98 127) 1.24 1.16 4.64 
5.41E+12 0.99 1.19 Lil 1.00 8.76 
6.14E+10 0.98 1.15 iY 1.00 7.26 
8.39E+11 0.97 22 1.24 1.00 1.70 
1.54E+12 0.97 AIS 1.16 1.00 1.39 
1.34E+11 0.97 122 PS 1S 6.17 
7.14E+11 0.99 Nel, 1.13 1.00 2.68 
4.14E+10 0.81 1.06 1.24 1.20 3.19 
1.21E+10 0.94 1.18 1.24 1.36 5.43 
2.86E+10 0.96 1.60 1.65 2.20 6.04 
1.93E+10 0.98 Ii 1.14 2.30 6.04 
2. 19E+10 0.96 1.26 1.30 39) 17 
4.36E+8 1.00 eb lesa 1.64 8.61 
2.27E+8 0.96 1.28 1.32 1.48 WS2. 
1.03E+8 0.96 46.6 46.6 USS 426 
9.33E+8 0.99 lee 1.13 152 1.88 
1.88E+9 0.97 1.20 || DY 1.19 2.29 
4.05E+10 0.81 1.06 1225 1.15 2.86 
1.13E+10 0.94 1.20 1.26 e382 5.07 
2.82E+10 0.96 1.40 1.44 oe 3.30 
1.62E+10 0.97 1.16 1.19 DBS) 6.69 
2.14E+10 0.96 1.26 1.30 2.94 10.8 
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Table 1 (continued) 7 
Variance estimates for Y with ratio adjustment in ACE-1 survey 


i Yu ge Vin Vy Vy 
Industry Item Year Y Vay a vi ae Yn vn 
339900 Total 2002 1.75E+6 1.94E+10 0.99 1277 2s) 1.10 3.71 
2003 1.58E+6 2.99E+10 0.98 1.24 Ih T) 1.10 1.60 
2004 1.70E+6 1.00E+10 0.99 1.40 1.40 1.69 4.61 
2005 1.77E+6 2.55E+10 0.99 1.28 1.29 25 3.02 
2006 1.94E+6 5.51E+10 0.99 1223 1e25 Lal 2S 
Structures 2002 2.99E+5 P21E+9 0.99 1.24 1.24 1.09 355 
2003 1.93E+5S 8.54E+8 0.99 1.27 1.28 1.09 Neg) 
2004 2.10E+5 2.00E+8 0.99 1.86 1.87 2.08 5.89 
2005 2.56E+5 5.07E+8 0.99 1.80 1.81 1.97 9.61 
2006 5.97E+5 4.93E+10 0.99 1.19 1.20 1.01 1.16 
Equipment 2002 1.45E+6 1.62E+10 0.99 727/ 1.28 1.07 3.02 
2003 1 .39E+6 2.71E+10 0.97 1.24 127 1.09 1.58 
2004 1.49E+6 9.14E+9 0.99 1.40 1.41 1.62 4.61 
2005 1. S1E+6 2.45E+10 0.99 il 2272 1.23 Les N72 
2006 1.34E+6 5.65E+9 0.99 1.42 1.43 1.60 6.20 
541300 Total 2002 3.38E+6 2.32E+10 0.99 1.47 1.48 1.67 5.02 
2003 3.09E+6 2.61E+10 0.99 1.26 [\e727) 1.05 1.62 
2004 3.97E+6 PA2E+11 1.00 1223 1223 1.03 1.37 
2005 4.94E+6 2.54E+11 1.00 1.20 1.20 1.04 1.71 
2006 4.96E+6 2.82E+10 1.00 1.40 1.40 ES 8.36 
Structures 2002 7. 41E+5 6.32E+9 1.00 1.70 afl 1.64 7.47 
2003 4.29E+5 3:32E+9 1.00 129 1.29 1.01 1.33 
2004 6.96E+5 4.38E+10 1.00 bay? eZ 1.00 1.40 
2005 TA2ES 9.00E+9 1.00 125 12S 1.08 2.08 
2006 8.73E+5 3.44E+9 1.00 1.58 19 1.63 9.88 
Equipment 2002 2.96E+6 1.39E+10 0.99 eS, 1.38 1.54 3.95 
2003 2.66E+6 1.94E+10 0.99 1.25 1.26 1.05 159 
2004 3.27E+6 5.83E+10 1.00 [ye 1.23 1.04 1.29 
2005 4.23E+6 2.40E+11 1.00 1.19 1.20 1.03 E59) 
2006 4.09E+6 2.35E+10 1.00 e277 1.28 1.49 5.47 
Table 2 
Population characteristics for the simulation study 
Population Sampling Correlation with Payroll 
Industry Stratum size fraction Total Structures Equipment 
211000 10 26 1.00 0.65 0.53 0.95 
2A 128 0.77 0.68 0.66 0.22 
2B 3) 0.11 0.57 0.51 0.51 
AC 1,800 0.02 -0.07 0.00 -0.10 
2D 10,406 0.00 0.28 0.00 0.28 
212300 10 30 1.00 0.96 0.95 0.94 
2A 108 0.37 0.85 0.74 0.77 
2B 414 0.07 0.03 0.76 -0.03 
AC 1,310 0.03 0.42 0.13 0.43 
2D 4,762 0.01 0.44 -0.22 0.44 
339900 10 158 1.00 0.80 0.40 0.80 
2A 498 0.26 0.40 0.04 0.51 
2B 2,048 0.05 0.20 0.24 0.18 
KC 6,310 0.02 0.19 0.48 0.09 
2D 25,288 0.00 0.37 0.67 0.36 
541300 10 160 1.00 0.60 0.56 0.59 
2A 959 0.38 0.20 0.39 0.06 
2B 4,531 0.06 0.28 0.13 0.27 
2C LETS 0.01 0.08 0.06 0.08 
2D 67,440 0.00 0.13 -0.01 0.15 
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Table 3 
Simulation results (in %) for variance estimation under covariate-dependent response mechanism 
Estimate Item Industry Vi Yi v, May v, vy 
Ratio Total 211000 RB -35.8 -38.1 -10.3 -8.0 39:1 ES29 
ST 49.8 50.4 47.4 48.6 252.9 182.9 
ER 19.6 19.8 ee? 11.8 10.7 1.1 
212300 RB -20.4 Elo 4.48 -2.69 54.8 266.4 
ST 30.3 Sl 26.8 ES} 139.1 268.8 
ER 12.6 12.6 DE, 9.6 6.3 0.1 
339900 RB -21.2 -22.5 0.26 ESS -5.34 25 
ST 47.3 47.0 55.0 56.0 43.9 67.8 
ER 14.3 14.6 10.4 10.3 10.0 2.6 
541300 RB -20.7 -21.0 3.83 4.08 -11.6 18.4 
ST 825) 32.8 34.9 3510 29.4 32.0 
ER 12.6 12.7 8.6 8.6 10.7 6.2 
Structures 211000 RB -38.0 -40.1 -11.9 -9.59 33.9 108.1 
ST Si Sled 48.5 49.6 244.4 180.8 
ER 20.9 Ziel 12.9 12.6 11.1 Leal 
212300 RB -23.2 -23.9 -12.4 -11.6 B32 341.5 
ST 31S 32.0 ANTI 27.0 9510 344.3 
ER 12.3 3 10.4 10.3 6.9 0.1 
339900 RB -20.0 -20.4 -6.31 -5.88 -10.9 39.8 
Si 42.5 42.7 42.3 42.3 39.9 64.0 
ER ISL) 16.0 aH 12.6 3:2 5.4 
541300 RB -20.0 -20.1 0.09 0.33 -15.9 len 
ST 42.6 42.5 50.5 50.7 41.1 42.7 
ER LSet 3) 7% on eo) 12.0 6.5 
Equipment 211000 RB -15.0 -17.3 14.1 16.4 -9.37 2129 
ST 63.9 62.6 87.7 90.0 64.1 69.6 
ER 16.2 16.7 1383 13.0 14.7 6.7 
212300 RB -21.4 -23.3 -4.13 -2.21 Boel 201.1 
ST BO 30) 28.4 29.0 113.7 204.4 
ER fer! 15 10.2 10.1 ed 0.2 
339900 RB -21.4 -22.8 1.18 eS) -7.29 50.8 
ST SZ 50.9 60.8 61.9 47.9 69.2 
ER 15.5 15.8 11.6 11.4 11.0 713} 
541300 RB -19.7 -19.9 6.16 6.43 -11.9 12.8 
ST 33.8 B39 38.4 38.5 31.0 30.9 
ER eS 12.5 8.9 8.9 11.0 7.0 
Count Total 211000 RB -30.1 -31.9 0.05 135) 1.05 103.1 
ST 50.4 50.5 Dey Shes! 46.7 113.4 
ER S33 15.6 9.0 8.8 8.7 1.0 
212300 RB -33.2 -34.6 -6.30 -4.96 17.6 204.5 
ST 38.7 39.6 Die 27.8 42.8 208.6 
ER 14.1 14.7 g). I 8.7 6.9 0.4 
339900 RB -23.9 -24.6 Les: 2.44 -14.2 46.4 
ST 47.5 47.4 Da pI) 43.4 62.4 
ER 13.4 SKS onl II 10.7 Des) 
541300 RB -22.9 -23.2 1.68 1.94 -18.8 15.4 
ST 328) 33.0 32.0 822 30.2 28.9 
ER UES) 11.6 UP eal 10.6 a2, 
Structures 211000 RB -30.3 -32.2 -0.15 1.65 -1.27 101.5 
ST Syl} Sls, les 58.7 46.7 112.3 
ER 15.8 16.3 9.6 9.4 SP) 0.8 
212300 RB -37.4 -38.0 -13.5 -12.9 3/538 250.2 
ST 41.6 42.0 28.9 28.8 S2ey 254.8 
Layee 15.4 15.6 9.6 oS 8.1 0.4 
339900 RB -20.0 -20.3 -4.33 -4.00 -14.5 38.6 
ST 42.3 42.4 42.4 42.4 40.1 62.8 
ER 14.6 14.7 THES 11.8 13.6 5.0 
541300 RB -20.9 -21.2 -0.54 -0.32 -18.9 14.5 
ST 41.6 41.6 47.8 48.0 40.6 40.9 
ER 125 5) Oa 9.1 Ven 6.0 
Equipment 211000 RB -17.8 -20.0 Wl 2 13.3 -13.0 26.6 
ST 58.9 58.0 76.4 78.4 Dieu 64.1 
ER NS.7/ 15.8 12:5 128) 14.5 6.1 
212300 RB -30.7 -32.2 -4.74 -3.27 12.1 164.3 
ST 37.6 38.6 291 2985) 38.7 168.9 
ER 14.1 14.5 9.6 DS eS 0.6 
339900 RB -24.] -24.9 De) S27) -15.2 45.0 
ST S12 Sis 61.0 61.5 47.7 64.1 
ER 14.8 LS 9 9.8 ES 25 
541300 RB -21.6 -21.9 4.10 4.39 -18.1 10.1 
ST 33.6 BI) i) 33.2 BOs BES 28.2 
ER vet sa Wc isl 10.3 a. 
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Table 4 
Simulation results (in %) for variance estimation under within-stratum uniform response mechanism 
ae i i se 
Estimate Item Industry Vv, Wa Vv, Lah v, v, 
Ratio Total 211000 RB -49.2 -50.4 -17.2 -16.0 89.2 138.4 
ST 55.4 56.1 43.7 43.9 310.5 258.3 
ER 26.9 Ix) 13.9 13.8 9.10 1.80 
212300 RB -5.42 -7.99 16.1 18.7 111.7 3372 
ST 28.9 28.5 37.4 39.6 WIP? 341.7 
ER 13.5 13.8 9.85 9.50 5.85 0.05 
339900 RB -9.37 -10.5 18.0 72 16.5 83.8 
ST 45.8 45.4 59.0 60.1 48.4 95.6 
ER 14.5 14.7 9.55 955 8.60 2.65 
541300 RB -8.83 -9.03 18.2 18.4 6.62 44.5 
ST 26.7 26.8 36.6 36.7 28.2 523 
ER 12.6 12.6 8.45 8.45 9.70 5.35 
Structures 211000 RB -52.6 -53.7 -19.2 -18.0 78.4 128.0 
Si 58.0 58.7 45.3 45.4 290.8 248.5 
ER 28.7 29.0 [ey 14.9 9.80 LDS 
212300 RB -16.2 -18.1 De LS, 63.5 356.2 
ST 32.0 32.4 S/ll 38.5 108.6 361.9 
ER [55 16.0 10.9 10.7 6.65 0.35 
339900 RB -13.2 -13.6 1349 14.3 LSS 54.9 
ST 47.8 47.8 S97 SD 46.3 82.7 
ER G2 17.2 12.6 WAS) 13.8 6.40 
541300 RB -8.9 -9.2 now 19.5 -2.22 36.0 
ST 39.4 3933 5357 54.0 38.6 Dog 
ER WX) 12.9 8.85 8.85 Wiles! 6.85 
Equipment 211000 RB -1.1 -2.75 DS 2S JI 12.8 60.1 
ST 64.4 63.2 88.6 90.3 TN 89.8 
ER 15.3 15.6 12.0 12.0 127 5.10 
212300 RB -6.3 -8.96 16.8 19.4 90.0 263.1 
ST 30.3 29.8 39.3 41.6 148.6 269.1 
ER Wa) 14.2 10.1 9.60 6.75 0.15 
339900 RB -8.84 -10.1 19.5 20.7 15.8 84.6 
Sil 50.8 50.3 65.7 66.9 52.9 98.8 
ER leat 1533 10.3 10.3 9.50 2.45 
541300 RB -6.89 -7.1 19.9 20.1 6.76 38.5 
ST 28.6 28.6 40.0 40.1 30.1 48.3 
ER 12.4 12.4 8.60 8.55 10.3 5.90 
Count Total 211000 RB -27.8 -29.0 14.2 15.4 16.3 149.5 
SI. 47.4 47.5 S35 54.2 44.4 158.1 
ER 16.0 16.2 8.30 8.30 7.45 1.85 
212300 RB -33.5 -34.9 Nos) 16.7 MBS, DALES 
Sil 40.0 40.9 38.5 3955 39.4 228.5 
ER 18.8 19% 9.80 9.65 8.55 1.90 
339900 RB -16.5 -17.1 20.2 20.8 4.21 Weed 
ST 45.1 44.0 SL 58.5 42.4 87.6 
ER 15.6 15.8 9.40 9.35 10.8 3.20 
541300 RB -9.61 -9.81 18.9 19.1 -0.77 45.0 
ST 26.5 26.5 36.0 36.1 24.7 See 
ER 12.4 12.4 8.55 8.55 Wes! 4.85 
Structures 211000 RB -27.5 -28.7 14.5 Sea 14.6 149.0 
ST 48.1 48.1 54.5 55.6 45.1 157.6 
ER 17.1 eS 9.05 9.00 8.50 2.05 
212300 RB -39.4 -40.4 11.6 12.6 10.1 238.5 
ST 44.8 45.5 38.8 39.6 S27 248.4 
ER 20.2 20.7 9.95 9.85 10.3 1.80 
339900 RB -14.2 -14.6 13.6 14.0 -3.55 53:5 
ST 47.1 47.0 513 57.6 45.1 80.5 
ER 17.6 17.7 12.1 A 14.7 6.30 
541300 RB -9.54 -9.76 20.0 20.2 -5.32 36.0 
ST 39a 39.0 53:3 3) 38.3 55.8 
ER 12.6 12.6 9.05 9.05 IL, 6.55 
Equipment 211000 RB -8.12 -9.64 Dep 24.2 1.56 54.3 
ST 58.0 57.1 76.2 Gilet OWES) 82.1 
ER 16.5 16.7 12.4 12.4 14.6 6.45 
212300 RB -28.5 -30.0 ial 18.6 ZS 189.7 
ST 37.6 38.4 40.9 42.0 38.2 198.9 
ER 18.1 18.5 9195 9.80 9.25 be 
339900 RB -15.8 -16.4 21.8 22.5 4.69 76.8 
ST 49.5 49.3 64.6 65.2 47.4 Otel 
ER 16.4 16.5 9.45 9.40 11.4 3.20 
541300 RB -7.53 -7.74 20.2 20.4 0.26 38.8 
ST 28.2 28.2 392 39.4 Pall Pe 48.0 
ER 12.7 Weed 8.30 8.25 Hees 5.65 
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Rescaled bootstrap for stratified multistage sampling 


John Preston '! 


Abstract 


In large scaled sample surveys it is common practice to employ stratified multistage designs where units are selected using 
simple random sampling without replacement at each stage. Variance estimation for these types of designs can be quite 
cumbersome to implement, particularly for non-linear estimators. Various bootstrap methods for variance estimation have 
been proposed, but most of these are restricted to single-stage designs or two-stage cluster designs. An extension of the 
rescaled bootstrap method (Rao and Wu 1988) to stratified multistage designs is proposed which can easily be extended to 
any number of stages. The proposed method is suitable for a wide range of reweighting techniques, including the general 
class of calibration estimators. A Monte Carlo simulation study was conducted to examine the performance of the proposed 


multistage rescaled bootstrap variance estimator. 


Key Words: Bootstrap; Calibration; Multistage sampling; Stratification; Variance estimation. 


1. Introduction 


Stratified multistage sampling designs are especially 
suited to large scaled sampled surveys because of the 
advantage of clustering collection effort. Various methods 
exist for variance estimation for these complex survey de- 
signs. The most commonly used methods are the linearization 
(or Taylor) method, and resampling methods, such as the 
jackknife, balance repeated replication and the bootstrap. The 
linearization method can be quite cumbersome to implement 
for complex survey designs as it requires the derivation of 
separate variance formulae for each non-linear estimator. 
Some approximations are normally required for the variance 
of non-linear functions, such as ratios and correlation and 
regression coefficients, and functionals, such as quantiles. 

On the other hand, the various resampling methods 
employ a single variance formulae for all estimators. The 
replication methods can reflect the effects of a wide range of 
reweighting techniques, including calibration, and adjust- 
ments due to provider non-response and population under- 
coverage. The jackknife and balance repeated replication 
methods are only applicable to stratified multistage designs 
where the clusters are sampled with replacement or the first- 
stage sampling fractions are negligible. A number of 
different bootstrap methods for finite population sampling 
have been proposed in the literature, including the with- 
replacement bootstrap (McCarthy and Snowden 1985), the 
rescaled bootstrap (Rao and Wu 1988), the mirror match 
bootstrap (Sitter 1992a), and the without-replacement 
bootstrap (Gross 1980; Bickel and Freedman 1984; Sitter 
1992b). A summary of these bootstrap methods can be found 
in Shao and Tu (1995). 

Most of these bootstrap methods are restricted to single- 
stage designs or multistage designs where the first-stage 
sampling units are selected with replacement or the 


first-stage sampling fractions are small in most strata. 
However, in many large scaled sample surveys it is common 
practice to employ highly stratified multistage designs where 
units are selected using simple random sampling without 
replacement at each stage. Some typical examples of these 
types of surveys are employer-employee surveys, such as the 
Survey of Employee Earnings and Hours (ABS 2008), and 
school-student surveys, such as the National Survey on the 
Use of Tobacco by Australian Secondary School Students 
(White and Hayman 2006). 

McCarthy and Snowden (1985) proposed an extension of 
their with-replacement bootstrap to two-stage sampling in 
the special case of equal cluster sizes and equal within cluster 
sample sizes, while Rao and Wu (1988) and Sitter (1992a) 
have given extensions of their rescaled bootstrap and mirror 
match bootstrap methods to two-stage cluster sampling. 
More recently, Funaoka, Saigo, Sitter and Toida (2006) 
proposed two Bernoulli-type bootstrap methods, the general 
Bernoulli bootstrap and the short cut Bernoulli bootstrap, 
which can easily handle multistage stratified designs where 
units are selected using simple random sampling without 
replacement at each stage. The general Bernoulli bootstrap 
has the advantage that it can handle any combination of 
sample sizes, but it requires a much larger number of random 
number generations than the short cut Bernoulli bootstrap. 

In this paper, an extension of the rescaled bootstrap 
procedure to stratified multistage sampling where units are 
selected using simple random sampling without replacement 
at each stage is proposed. In Section 2, the notation for 
stratified multistage sampling is introduced. In Section 3, the 
extension of the rescaled bootstrap estimator to multistage 
sampling is described. The main findings of a simulation 
study are reported in Section 4. Some concluding remarks 
are provided in Section 5. 
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2. Stratified multistage sampling 


For simplicity, the case of stratified three-stage sampling 
is presented. Consider a finite population U divided into 
H nonoverlapping strata U = {U,, ..., U,,}, where U, is 
comprised of N,, primary sampling units (PSU’s). At the 
first-stage, a simple random sample without replacement 
(SRSWOR) of 1,, PSU’s are selected with selection 
probabilities 7,,, =7,,/N,, within each stratum A. 
Suppose selected PSU i in stratum / is comprised of N,,, 
secondary sampling units (SSU’s). At the second-stage, a 
SRSWOR of size n,,, SSU’s are selected with selection 
probabilities 1,,, = M,;/N>,; Within each selected PSU. 
Suppose selected SSU j in selected PSU 7 in stratum / is 
comprised of N,,,, ultimate sampling units (USU’s). At the 
third-stage, a SRSWOR of size n,,,, USU’s are selected 
with selection probabilities 75,,,. = 13, /N3,, within each 
selected SSU. 

The objective is to estimate the population total Y = 
Se ieee > ye Ynix» Where y,,, is the value for the 
variable of interest y for USU k in SSU 7 in PSU 7 in 
stratum A. An unbiased estimate of Y is given by: 


PS Sip) poy habeas) Seauioy 
iS V hiik 


h=l h=l Mp i=l Moni j=) “sny k=l 


where = (Ny, fy) DY, LN Nyy) LAY ny 
and ee 2 My [Msny) Ze Jn This estimator can also 
be written as Y=>7 yo 8 yy Wri Vagks Where Wyn = 
WiniW nig M3nijk = (Ni, 14) Noni ! Ayn; )N 3p / Ny; ) is the 
sampling weight for USU k in SSU 7 in PSU i in 
stratum /h. 

An unbiased estimate of Var(Y) is given by (Sarndal, 
Swensson and Wretman 1992): 


- ‘ a Ni 2 
Var(Y) = ) —* (1 - fy) 5, 
h=l Np 
Nin Non 2 
ee he = (hx fons) Son5 
h=l Np, i=l Mop; 
ti it N, i oC Nee 
a Dieser it - me : (i= den aks (2.1) 
h=l Np i=l Mon, j=l "5ny 
igs Sir= (m,/Ni,), Soni = Moni! Noni)» Fini = (71345! N3yi)> 


= yi Yili Y,, = Ti pha 1 Yoalans Vitina Lie 1 Yagk! Many 
-¥,)° /(n,,-1), Sous fa Glos, all /(Ny,; —1) 
and oot a at (Vrik i Vay t ty a1): 
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3. Rescaled bootstrap for stratified 
multistage sampling 


Rao and Wu (1988) proposed a rescaling of the standard 
bootstrap method for various sampling designs including 
stratified sampling. Since the rescaling factors are applied to 
the survey data values, this method is only applicable to 
smooth statistics. Rao, Wu and Yue (1992) presented a 
modification to this rescaled bootstrap method where the 
rescaling factors are applied to the survey weights, rather 
than the survey data values. This modified rescaled 
bootstrap method is equivalent to the orginal rescaled 
bootstrap method, but has the added advantage that it is 
applicable to non-smooth statistics as well as smooth 
statistics. Kovar, Rao and Wu (1988) showed that when 
using a bootstrap sample size of n, =n, —1 the rescaled 
bootstrap estimator performed well for smooth statistics. 

Although bootstrap samples are usually selected with 
replacement, Chipperfield and Preston (2007) modified the 
rescaled bootstrap method to the situation where the 
bootstrap samples are selected without replacement. Under 
this without replacement rescaled bootstrap method it can be 
shown that the choice of either n, =|n,/2| or n, = 
[7,/2] is optimal, where the operators |x| and [x] 
round the argument x down and up respectively to the 
nearest integer. The choice of n, = ERA has_ the 
desirable property that the bootstrap weights will never be 
negative. 

For simplicity, the case of stratified three-stage sampling 
is presented, but the proposed procedure can easily be 
extended to any number of stages. The without replacement 
rescaled bootstrap procedure for stratified three-stage 
sampling is as follows: 


(a) Draw a simple random sample of n,, PSU’s without 
replacement from the n,, PSU’s in the sample. Let 6,,, be 
equal to 1 if PSU i in stratum A is selected and 0 
otherwise. Calculate the PSU bootstrap weights: 


s n 
pe! 1h 
Water mat —)i, + An, "3 3 


lh 


n,, (1 — Sin), - Tay )- 

(b) Within each of the PSU’s in the sample, draw a simple 
random sample of n;,, SSU’s without replacement from the 
n,; SSU’s in the sample. Let 5,,, be equal to 1 if SSU 7 
in PSU i in stratum /h is selected and 0 otherwise. 
Calculate the conditional SSU bootstrap weights: 


where i,, = 
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* 
Wonig = 
Ww, 
lhi 1 
Whig + [i — Rip t+ Aye Oy, 
Wihi lh 
n n td Pape 
1h 1h 2hi 
NOW| ala CORA 5 Oks = Oona 
Nyy, lh 2hi 
* * 
where A445 = J Moni SinG@ — Soni )/Coni — Moni): 


(c) Within each of the SSU’s in the sample, draw a simple 
random sample of N3pi USU’s without replacement from 
the 7;,,, USU’s in the sample. Let 5,,,, be equal to 1 if 
USU k inSSU 7 in PSU i in stratum /A is selected and 0 
otherwise. Calculate the conditional USU bootstrap weights: 


= 
Wspijk = 


Wins “2h: 

Lhi Zhi 

Wnuk a7 re Nee as me =i 
Wihi Wo ij 


n n 
hs 2h, 
Paw | =a + dans) oe Oth ne a ony 
Nh, Np Ny pj 
| n | Np ,. 
lh 2hi 
Dshij ROL | ae Oo hij 
Np Ny hj 
aX Mal “ni § Bhi 
Shij Shi Soni ay O3 nik 
Ny pj N3 pi; 
* 
= N34 )- 


ee Gites Faniz)! nj 


ee iin Mane Matas - 3 
1 Dy DS > Whijk Y hiik? 0 = g(Y ) 
‘Srey re 


where Whisk = Wins Wonii Wahi: 
(e) Independently repeat steps (a) to (d) a large number of 


times, B, and calculate the bootstrap estimates, Q) 
6? 6. 


(f) The bootstrap variance estimator of 6 is given by: 


Var (6) = £.(6 — E.(6))’ (3.1) 


or the Monte Carlo approximation: 


ape Ao) _ By? 
Var (6) = ecu 2 (8 0) 
where 6 = Y2_,67/B. 

It is shown in the Appendix that the multistage rescaled 
bootstrap variance estimator for stratified three-stage 
sampling as defined by (3.1) reduces to the standard 
unbiased three-stage variance estimator (2.1) in the case of 
6 being a linear estimator. The choice of 1, =|n,,/2], 
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Nyy; = | My/2| and 73, = | M5 /2| will be optimal and 
will have the desirable property that the bootstrap weights 
will never be negative. 

The proposed procedure can easily be extended to 
any number of stages by adding terms of the form 


re Tea (n, /n2)5,) +2, TeV (n, (nt) *\ ip) Ny Oo, 
at ch stage, R, to a bootstrap weight sien where 
MAS oh Ts A Fpl (m, ni), 

Yeo, Mantel and Liu (1999) presented an enhancement 
to the rescaled bootstrap which accounted for adjustments 
made to the design weights, such as post-stratification. For 
example, consider a simple case of non-integrated cali- 
bration using auxiliary information for two-stage stratified 
sampling (Estevao and Saérndal 2006), which has the dual 
objectives of producing estimates for both a first-stage 
variable of interest Y, = Yiniycu jp,; aS well as a second- 
stage variable of interest, Y, = D(nijyeu Van. Assume there 
exists: 


(i) a set of p first-stage auxiliary variables x,,, for which 
the population totals X, = YuicuX,; are known, and 
where the population totals are generated from a list frame 
of PSU’s for which the x,,, are known for every PSU in the 
population; and 

(ii) a set of q second-stage auxiliary variables x,,,, for 
which the population totals X, = Yirijyceu X2, are known, 
where the population totals are acquired from an external 
source. 


The auxiliary variables can be used to form the first-stage 
and second-stage calibration estimators: 


Youu = >) Wii Vth 


(hi)es, 


Yoat2 = SS Won 2ni 


(hij Jes, 


where the first-stage calibration weights, w,,,, and the 
combined first-stage and second-stage calibration weights, 
Wrap» are given by: 


T 
Wii = Wri ; + [x in >» rh 


(hi)es, 


a 
i 
yi WinidiniM ni | thi 
(hi)es, 


Won — “in ony ; ap x, Tr » 


(hij Es, 


-1 
T 
Ss WiniW2nij X2nipX2nij | X2nii | 
(hij Es, 
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Then the multistage rescaled bootstrap method can easily 
be modified in a similar manner to handle these calibration 
estimators by replacing step (d) in the procedure as follows: 


(d) Calculate the first-stage and second-stage calibrated 
bootstrap weights in the same manner as the first-stage and 
second-stage calibrated weights: 


_* * * 
Wai = Wiani| 1+} X, - »: Wii™ hi 


(hi)es, 


_* * * 1 xX * * 
Wrong = Miniony | + + | Ads » Wii 2nij X 2h 
(hij es, 


* * ff 
ae WiniW2nij X2nij X 2h 
(hij es, 


X 4 hij 


The first-stage and second-stage calibrated bootstrap 
estimates are calculated as: 


A* 2s 
Yoatt = a Wii D thi 


(hi)es, 


raw 
II 


Pri. 
CAL2 > Wi2nij Y2nij° 
(hifyes, 


This procedure can easily be modified to any type of 
calibration and extended to any number of stages. This 
modification of the rescaled bootstrap takes into account 
adjustments made to the design weights due to calibration. 
Ideally all adjustments made to the design weights, 
including adjustments due to provider non- response and 
population under-coverage should also be made to the 
bootstrap weights. 


4. Simulation study 


A Monte Carlo simulation study was conducted to 
examine the performance of the multistage rescaled 
bootstrap variance estimator. The study was restricted to 
stratified two-stage sampling. The simulation study was 
based on ten artificial populations, each of which was 
stratified into H =5 strata, with N,, = 50 first-stage units 
within each stratum, and N,,, =40 second-stage units 
within each first-stage unit. 

Firstly, the first-stage auxiliary variable x,,, for each 
first-stage unit i in stratum h was generated from the 
normal distribution N(1,1,. (1 — Py) Or14/Pyp)- Sec- 
ondly, the second-stage auxiliary variable, x,,,, and the 
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second-stage target variables, y,,, and Z,,,, for each 
second-stage unit j within first-stage unit i in stratum h 
were then generated from the multivariate normal 
distribution N, (1,,;, Z>,;) Where p,,, is the mean vector: 


Hy ohi 
Boni = | By 2ni 
Ho 2hi 


with § Hyo_; = Hyon = Bian = “nie and %,,, 
variance-covariance matrix: 


2 


Oy 2hi Pry 2niPx2niP yrni — Pxz2Ai Px 281 Oz 2h: 


2 


Loni =| Pay2ni Px 201 P y2hi 0 y2hi Pyz2niP y2niP z2hi 


2 


PrerniPx2niFz2hi — Pyz2ni Py2niPz2hi © 2 2hi 


with O74; = Sym = Oran = (1 — Pwami) Swans / Pwonr 

The parameter values that were kept stable across all ten 
populations were p,,, = 25x (h +1), of, =10, 02 ,; = 
100, Pyar = Pxzre =9.75 and p,,,,; =0.50. The para- 
meter values that were varied across the ten populations 
were f,, the first-stage sampling fractions, /,,;, the 
second-stage sampling fractions, p,,, and p,,,;. These 
parameter values are presented in Table 1. 


Table 1 
Characteristics of simulation populations 
Sin Sani Pp Pw 
Pop I 0.1 0.1 0.75 OF/> 
Pop II 0.1 0.1 0.25 0.75 
Pop III 0.1 0.5 0.75 0.75 
Pop IV 0.1 0.5 0.25 0.75 
Pop V 0.1 0.5 0.25 0.25 
Pop VI 0.5 0.1 0.75 0.75 
Pop VII 0.5 0.1 0.75 0.25 
Pop VIII 0.5 0.1 0.25 0.25 
Pop Ix 0.3 0.3 0.75 0.25 
Pop X 0.3 0.3 0.25 0.25 


The parameters of interest used in the simulation study 
were the population mean, p1,, the population ratio, R,, = 
H,/p,, the population correlation coefficient, p,, = 
6,,/5,6,, the population regression coefficient, B,, 
oes O;,, and the population median, M,. 

In order to estimate these parameters of interest using the 
multistage bootstrap variance estimators, a total of S = 
20,000 independent two-stage simple random samples were 
selected without replacement from each of the ten artificial 
populations. In addition, a grand total of T = 100,000 
independent two-stage simple random samples were selected 
without replacement from each of the ten artificial 
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populations in order to estimate the true population variances 
for the parameters of interest. The multistage bootstrap 
variance estimators were calculated using B = 100 bootstrap 
samples. 

The accuracy of the multistage bootstrap variance 
estimators were compared using the relative biases (RB) and 
the relative root mean square error (RRMSE). These 
measures were calculated as: 


2 ix 7) War(P 
RB = ee 2 (Var. (Y, ) vart?| 
1 [1 
RRMSE = Va 5 Swart?) =WVartn)): 


where Var(Y) = 7 'Y7,(¥, — Y)’ is the estimated true 
population variance, and Var.(Y,) are the multistage 
bootstrap variance estimators for the s" simulation sample. 

The multistage rescaled bootstrap variance estimator 
(MRBE) was compared against the single-stage rescaled 
bootstrap variance estimator (SRBE) and the multistage 
general Bernoulli bootstrap variance estimator (BBE) 
proposed by Funaoka et al. (2006), with bootstrap samples 
using the non-calibration estimation weights, w,, = 
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W,;W>,,; The relative biases and relative root mean square 
errors of MRBE, SRBE and BBE using the non-calibration 
estimation weights for the ten artificial populations are 
given in Tables 2 and 3. 

In the case of linear functions, such as means, and non- 
linear functions, such as ratios, correlation coefficients and 
regression coefficients, the MRBE performed better than the 
SRBE and BBE with respect to relative bias and relative 
root mean square error. While the MRBE performed 
consistently well across all ten artificial populations, the 
SRBE only performed well for artificial populations III, TV 
and V, where the first-stage sampling fractions were small 
(f,, = 9.1) and the second-stage sampling fractions were 
large (f,,; =9.5), and the BBE only performed well for 
artificial populations VI, VII and VIII, where the first-stage 
sampling fractions were large (f,, = 0.5) and the second- 
stage sampling fractions were small (/,,,; =0.1). These 
sampling fractions were similar to the first-stage and 
second-stage sampling fractions used in the simulation 
study presented in Funaoka etal. (2006). The different 
levels of correlation between the first-stage units, and 
between the second-stage units within the first-stage units, 
controlled by varying the parameters p, and p,, had little 
impact on the performance of the variance estimators. 


Table 2 
Relative bias (Yo) of variance estimators 
Mean (Hy) Mean (y1,) Ratio (Ry) 

MRBE SRBE BBE MRBE SRBE BBE MRBE SRBE BBE 
Pop I -0.28 -6.73 27.10 0.42 -6.63 2732 0.00 -9.07 36.22 
Pop II -0.05 -2.21 11.83 0.59 -1.64 12.54 -0.43 -9.26 36.40 
Pop Ill -0.79 -2.63 3.62 -0.93 -2.66 3.40 -0.17 -5.30 Syily 
Pop IV -0.23 -0.52 3.60 -0.18 -0.46 3.61 0.53 -4.65 5.98 
Pop V 0.15 -1.60 4.55 0.15 -1.64 4.54 0.52 -4.85 6.41 
Pop VI 0.70 -39.18 -0.34 0.65 -39.36 -0.28 1ST -46.40 1.30 
Pop VII 0.19 -46.19 -0.26 -0.06 -46.48 -0.57 -0.27 -48.19 -0.73 
Pop VIII 0.37 -38.62 -0.41 0.23 -39.36 -0.46 -0.26 -47.93 -0.62 
Pop IX 0.42 -20.85 -7.76 -0.51 -20.03 -8.41 0.13 -23.13 -8.87 
Pop X -0.56 -12.35 -6.08 0.70 -10.87 -6.93 -0.72 -23.70 -9.5] 

Correlation Regression 
Coefficient (Py) Coefficient (B,.) Median (M,) 

MRBE SRBE BBE MRBE SRBE BBE MRBE SRBE BBE 
Pop I -2.3] -10.23 B2ali7) -0.08 -9.05 36.41 19.04 -19.86 Bal 
Pop II -1.51 -8.4] 29.65 0.05 -8.74 36.41 19.29 2.42 40.85 
Pop Ill 0.36 -4.37 5.69 0.05 -5.12 5.42 7.50 4.28 9.72 
Pop IV 2.18 -0.60 ele, 0.28 -5.05 5.70 17.40 16.17 34.37 
Pop V 0.79 -2.7]1 5.95 0.26 -5.40 6.34 8.29 4.78 11.49 
Pop VI 0.32 -46.67 0.14 0.89 -46.59 0.69 1B. S7 -33.56 9.15 
Pop VII -0.07 -46.78 -0.39 -0.21 -47.85 -0.60 14.68 -38.16 11.86 
Pop VIII 0.31 -44.25 -0.27 -0.09 -47.54 -0.55 2.09 -38.90 -0.64 
Pop Ix -0.93 -23.02 -9.30 -0.20 -23.48 -9.20 8.08 -17.23 -1.97 
Pop X -0.82 -19.35 -8.24 -1.02 -23.89 -9.75 2.10 -13.84 -5.46 


Note: The largest simulation error on the relative biases was less than 0.7%. 
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In the case of non-smooth statistics, such as medians, 
both the MRBE and the BBE tended to overestimate the 
true population variances, while the SRBE tended to 
underestimate the true population variances. Furthermore, 
the relative root mean square errors for medians were up 
to 3 times larger than the relative root mean square errors 
for means. The MRBE performed better than the BBE for 
the artificial populations I to V where the first-stage 
sampling fractions were smaller (f,, = 9.1), while the 
BBE performed slightly better than the MRBE for the 
artificial populations VI to X where the first-stage 
sampling fractions were larger (f,, = 0.3 or 0.5). 

This overestimation of the multistage rescaled boot- 
strap for medians was similar to the findings shown in the 
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studies by Kovar et al. (1988) and Rao etal. (1992) for 
the single-stage rescaled bootstrap. It should be noted that 
the original rescaled bootstrap introduced by Rao and Wu 
(1988) was developed only for smooth statistics, such as 
means, ratios, and correlation and regression coefficients. 
The MRBE was examined using the calibration 
estimation weights, W,,, = W,,;W,,, Which satisfy the 
calibration constraint )i(pi;)es, WiniVoniX2ny = X2, Where 
X, = Laie X24 18 the population total for the second- 
stage auxiliary variable. The relative biases and relative 
root mean square errors of the MRBE using the 
calibration estimation weights for the four artificial 
populations II, IV, VII and IX are given in Table 4. 


Table 3 
Relative root mean square error (%) of variance estimators 

Mean (p,) 

MRBE SRBE BBE MRBE 

Pop I SS, 32.1 44.6 Sled 
Pop I 33.9 33.8 38.2 33.4 
Pop Ill 33.8 33.8 35.9 33.0 
Pop IV 33) Boe 37.4 B52 
Pop V 32.0 Bio 34.2 34.3 
Pop VI 16.4 40.7 16.5 16.4 
Pop VII 16.1 47.4 16.4 16.1 
Pop VIII 16.3 40.3 16.5 16.7 
Pop Ix Ne) 26.7 20.0 ONS 
Pop X 19.8 22.4 202 I) 

Correlation 


Coefficient (p,,) 


MRBE SRBE BBE MRBE 
Pop I 47.8 46.3 68.7 36.6 
Pop II 48.4 47.1 66.6 37.4 
Pop Il 859. 35.6 38.4 37.5 
Pop IV 42.6 42.2 45.4 38.0 
Pop V 40.3 40.0 43.3 SYLS 
Pop VI 21.6 48.4 DNA 16.9 
Pop VII 21.4 48.4 AES 16.9 
Pop VIII 21.6 46.3 21.5 17.0 
Pop Ix DAES 29.4 22 20.5 
Pop X 22a 27.8 23.4 20.6 


Coefficient (B,.) 


Mean (1,) Ratio (Ryz) 
SRBE BBE MRBE SRBE BBE 
31.8 44.4 31.8 B23 51.4 
3383 38.1 B22 329 Sih7 
33.0 35.0 33.0 B35] 35a) 
35.2 SMe8) 32.8 32.8 35.0 
34.2 36.5 33.0 33.1 35.6 
40.9 16.5 16.5 47.5 16.5 
47.8 16.4 16.1 49.0 16.1 
40.9 16.3 16.2 48.8 16.1 
26.3 20.0 19.2 28.6 20.2 
21.6 20.3 19.1 29.0 20.6 

Regression 


Median (M,) 


SRBE BBE MRBE SRBE BBE 
372 Spe! 88.7 80.1 89.8 
37129 55.6 93.4 91.0 tS 9) 
37.6 399 80.4 80.3 81.1 
38.0 40.3 1D 96.6 2S 
SMS) 40.1 315 30.7 63.3 
47.8 17.0 55:3 51.4 52.0 
49.0 16.8 555 51.4 51.4 
48.6 16.9 41.8 49.7 40.3 
299) 21.6 46.1 42.7 42.7 
30.2 ZAeo BOW 38.9 3719) 


Table 4 
Relative bias (%) and relative root mean square error (“%) of rescaled bootstrap variance estimator 
Hy Ry; Pyz By: My 
Relative Bias (%) 
Pop II -0.42 -0.29 -1.51 -0.08 20.98 
Pop IV 0.40 0.49 1.83 0.08 18.28 
Pop VII -0.22 -0.24 -0.03 -0.28 12.24 
Pop IX 0.62 0.19 -1.00 -0.20 7.24 
Relative Root Mean Square Error (%) 
Pop II 32.6 32.4 48.4 313 97.8 
Pop IV 32.8 32.8 44.6 379 99.4 
Pop VII 16.2 16.1 MNS 16.9 50.0 
Pop IX 19.1 LO 21.6 20.5 43.8 


Note: The largest simulation error on the relative biases was less than 0.6%. 
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The relative biases and relative root mean square errors of 
the MRBE using the calibration estimation weights were 
similar to those using the non-calibration estimation weights. 


5. Conclusion 


This paper extends the rescaled bootstrap procedure to 
multistage sampling where units are selected using simple 
random sampling without replacement at each stage. Under 
the proposed multistage rescaled bootstrap method, the 
bootstrap samples are selected without replacement and 
rescaling factors are applied to the survey weights. This 
proposed method is relatively simple to implement and 
requires considerably less random number generations than 
the multistage general Bernoulli bootstrap method. The 
proposed method is also suitable for a wide range of 
reweighting techniques, including calibration, and adjust- 
ments due to provider non-response and population under- 
coverage. Furthermore, the results of the Monte Carlo 
simulation study indicate that the multistage rescaled boot- 
strap performs much better than the single-stage rescaled 
bootstrap and the multistage Bernoulli bootstrap for smooth 
statistics, such as means, ratios, and correlation and re- 
gression coefficients. 


Appendix 


In this Appendix it is shown that the multistage rescaled 
bootstrap variance estimator for stratified three-stage sam- 
pling reduces to the standard unbiased three-stage variance 
omen (2.1) in the case of 6 being the linear estimator, 

pa als it Whe Yaga: ~The bootstrap variance 
estimator of Y° . given by: 


Var(Y’) = Var, (E,.(E(Y_))) 
+ E,,(Var,.(E,.(Y'))) + £,.(E.(Var, (Y"))). 


Using standard results on the expectation and variance 
with respect to the SRSWOR bootstrap sampling and some 
tedious but straightforward algebra, the components of 
bootstrap variance estimator are given below. The 
conditional expectation of Y" given Sis 


E,.(Y ) = 

H my, Mri 

>y Wi Wy | 1 — Arp self = Oi 
h=1 i=l j=! Mp, 


n n Vinge A 
lh lh 2hi 

a oni * ini + Aon * Sthi * Sony Y; 
Vp Np Ny pj 
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and the conditional variance of Y" given s, is 
Ni Mh N,, Ny pj 2 


N3 pi 
i 2 
ne hi So hi- Ory Fn) Sn: 


h=1 Ny, i=l Thy j= N3pij 


The conditional expectation of i286 0% ") and Var. (Y 5) 
given s, are 


< ET n 
E,.(E3.(¥ ) = sat >> w - Mn Bu: Mine — Sini) Lis 
h=1 i=l ni, 
E,.(Var,.(Y )) = 
e Ny, Noni a Non 2 
de rs oy > Shi i Soni) S3hij 


hal yp, izy Mop j= N3 pi 


and the conditional variance of E,.(Y") given s, is 


Vary y= i Mu 3 Ohi New (l Set YS). 


h=I Ny, i=] Ny pj 


Finally, the conditional expectation of E,.(Vaty.(¥ *)) 
and Var,.(Ey.(Y")) given s, are 


E «(Ey (Var,.(Y" ))) = 
Hf Mh N. Poni 2 F 
ea 2hi Sy 3hij 0 ee 
~. My al Moy Ga Many 3hij ) hij 
Fate (lea e= > = - PRN 
as Pal ht 


which are equal to the third and second terms of (2.1) 
respectively, and the conditional variance of E,,.(E;.(Y *)) 
given s, iS 


H 2 
Var,.(E,.(E3.(Y ))) = > = dai) i, 
h=) lh 


which is equal to the first term of (2.1). 
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Use of within-primary-sample-unit variances to assess 
the stability of a standard design-based variance estimator 


Donsig Jang and John L. Eltinge ' 


Abstract 


In analysis of sample survey data, degrees-of-freedom quantities are often used to assess the stability of design-based 
variance estimators. For example, these degrees-of-freedom values are used in construction of confidence intervals based on 
t distribution approximations; and of related ¢ tests. In addition, a small degrees-of-freedom term provides a qualitative 
indication of the possible limitations of a given variance estimator in a specific application. Degrees-of-freedom calculations 
sometimes are based on forms of the Satterthwaite approximation. These Satterthwaite-based calculations depend primarily 
on the relative magnitudes of stratum-level variances. However, for designs involving a small number of primary units 
selected per stratum, standard stratum-level variance estimators provide limited information on the true stratum variances. 
For such cases, customary Satterthwaite-based calculations can be problematic, especially in analyses for subpopulations 
that are concentrated in a relatively small number of strata. To address this problem, this paper uses estimated within- 
primary-sample-unit (within PSU) variances to provide auxiliary information regarding the relative magnitudes of the 
overall stratum-level variances. Analytic results indicate that the resulting degrees-of-freedom estimator will be better than 
modified Satterthwaite-type estimators provided: (a) the overall stratum-level variances are approximately proportional to 
the corresponding within-stratum variances; and (b) the variances of the within-PSU variance estimators are relatively small. 
In addition, this paper develops errors-in-variables methods that can be used to check conditions (a) and (b) empirically. For 
these model checks, we develop simulation-based reference distributions, which differ substantially from reference 
distributions based on customary large-sample normal approximations. The proposed methods are applied to four variables 
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from the U.S. Third National Health and Nutrition Examination Survey (NHANES III). 


Key Words: Complex sample design; Degrees 


of freedom; Errors-in-variables 


regression; Satterthwaite 


approximation; Stratified multistage sample survey; Two-PSU-per-stratum design; U.S. Third National 
Health and Nutritional Examination Survey (NHANES III). 


1. Introduction 


1.1 Motivating example: Inference for special 
subpopulations in NHANES III 


This work arose from a study of inference for geograph- 
ically concentrated subpopulations in the U.S. Third National 
Health and Nutrition Examination Survey (NHANES III). 
For some general background on NHANES III, see National 
Center for Health Statistics (1996). In many analyses, 
NHANES III data are treated as arising from a stratified 
multistage sample design that uses 49 strata and two primary 
sample units (PSUs) per stratum. Consequently, formal 
inferences from NHANES III data (e.g., construction of 
confidence intervals) often use the assumption that the 
associated variance estimators are based on approximately 
49 degrees of freedom and are thus relatively stable. 

However, the Mexican-American subpopulation is 
concentrated in a relatively small number of strata, so 
associated variance estimators may be less stable (7.e., have 
greater sampling variability) than would be indicated by the 
nominal 49 degrees of freedom term. Consequently, it is 
important to use an appropriate estimator of the true degrees 
of freedom associated with variance estimators for such 


subpopulations, and to modify confidence interval calcu- 
lations accordingly. Development of an appropriate degrees- 
of-freedom estimator can be complicated by moderate or 
severe heterogeneity in the underlying  stratum-level 
variances. Such complications arose in the analysis of the 
four NHANES III variables listed in Table 1.1. Section 5 will 
consider inference for the means of these four variables for 
the subpopulation of Mexican-Americans aged 20-29. 


Table 1.1 
Four NHANES III variables 


Variable Name Description 

BMPWT Weight (kg) 

HAR3 Do you smoke cigarettes now? (0/1) 
TCRESULT Serum total cholesterol (mg/dL) 
HDRESULT HDL cholesterol (mg/dL) 


1.2 Stability of design-based variance estimators 


Suppose we have a population partitioned into L strata, 
with N, PSUs in stratum h for h = 1, 2, ..., L. Under a 


1. Donsig Jang, Mathematica Policy Research, 600 Maryland Avenue SW, Suite 550, Washington, DC 20024-2512, U.S.A. E-mail: DJang@Mathematica- 
mpr.com; John L Eltinge, U.S. Bureau of Labor Statistics, PSB 1950, 2 Massachusetts Avenue NE, Washington, DC 20212-0001, U.S.A. 
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stratified multistage sampling design, we select n, PSUs, 
with replacement, and with per-draw selection probability 
Pp, for PSU i within stratum h where yp, 1, Thus, 
a total of n = Yy_,n, PSUs are selected. Within selected 
PSU (A, i), n,, secondary sample units (SSUs) are selected 
with replacement and with per-draw selection probabilities 
Pri» Where ae Phy = 1 and N,, 1s the number of SSUs in 
PSU (Api) For’a*given’survey item; let" Y, be the 
population total for stratum h, and define the overall 
population total Y = )%_,Y,. The total Y may correspond 
to a total either for the full population or for a specified 
subpopulation. 

Our goal is to construct a confidence interval for the total 
TY Pivet ep be an unbiased estimator of Y,,,, the population 
total for secondary unit 7 in primary unit 7 in stratum A. 
Then a customary design-based estimator of Y is Y = 
Lia, where Y=) Lis Palys Prat is a design 
unbiased estimator of Y, based on data obtained from PSU 
j in stratum h; and Y,, = n,; 07" repeion is an unbiased 
estimator of Y,,, the population total for PSU i in 
stratum h. 

Under the standard condition that sampling is inde- 
pendent across strata, the variance of Y can be written, 
V(Y) = >i4V, where V, = Var(¥,). Throughout the 
remainder of this paper, we will call the V, terms the 
stratum-level variances, and we will assume that n, 2 2 for 
all h =1, 2, ..., L. Note that V, depends on the sample 
design used within stratum h, and is distinct from the 
within-stratum variance of element-level Y values. A 
simple unbiased estimator for V(Y) is V(Y) = XLV, 
where V, =n,'(n, —1) D7 (Betas * eae see, €8., 
Wolter (1985, page 44). Note that the estimator V, is a 
multiple of a sum of squared differences among the terms 
p,,Y,;. In addition, under regularity conditions the random 
variables p,,'Y,, will be approximately normally distributed 
for a given stratum h. Consequently, the overall stratum- 
level variance estimators V, generally will approximately 
satisfy the following condition. 


(Cl) Fork =1,°2)5.°L, the terms’V,"(n, —1)V, are 
distributed as independent chi-square random 


variables with n,—1 degrees of freedom, 
respectively, where n, = 2. 


Under condition (C.1), {V(Y)}'dV(Y) has the same 
first and second moments as a chi-square random variable 
with d degrees of freedom, where d is the solution to the 
equation, 

24M Y)}° -ViM(Y} d = 0 (1.1) 


or equivalently 


def 


d= (n, - A {V(Y)}? (1.2) 
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where V{V(Y)} = Y4_,2(n, -1)'V,. Direct substitution 
of V, for V, and V(¥) for V(Y) in expression (1.2) leads 
to the Satterthwaite (1946)-type degrees-of-freedom 
estimator, 


L =] 

d, = [de . 72 | VO)P sale) 
h=| 

For some general background on d s and related estimators, 

see, e.g., Smith (1936), Satterthwaite (1941, 1946), Cochran 

(1977, page 96) and Kendall, Stuart and Ord (1983, pages 

91-92). In constructing confidence intervals for a 

subpopulation parameter, Casady, Dorfman and Wang 

(1998) use Bayesian ideas to develop related degrees-of- 

freedom measures for a Student’s ¢ -statistic. 

For designs in which n, is large for all 4, the error in 
estimation of V, is relatively small, and d, can provide a 
satisfactory estimator of expression (1.2). However, many 
large-scale surveys use small n,, e.g., n, = 2. For small- 
n, cases, condition (C.1) and routine algebra lead to the 
expectation result E(V,) =(n, —1)'(n, +1)V,. This 
implies that the standard Satterthwaite degrees-of-freedom 
estimator d, can severely underestimate d, and that the cor- 
responding confidence interval Y + t Dee elas yy? may 
have a true coverage rate substantially below the nominal 
rate 1—a. Consequently, Jang (1996) considered an 
alternative degrees-of-freedom estimator, 


d= GE-14y (Oh) de (1.4) 


for the two-PSUs-per-stratum design. 


1.3 Use of auxiliary stratum-level data 


For cases in which there is moderate heterogeneity 
among the V, terms, simulation work by Jang (1996) 
indicated that Gan performs relatively well. However, if 
there is substantial heterogeneity among the stratum 
variances (i.e., if L''d is relatively small), then d,s may be 
unsatisfactory. The fundamental problem is that when the 
n, values are relatively small, the estimators V,, by 
themselves, do not provide sufficient information regarding 
the relative magnitudes of the true stratum-level variances 
V,. In some cases, a variance estimator based on auxiliary 
data is expected to be more stable than the customary 
design-based estimator; see e.g., Isaki (1983). Similarly, 
auxiliary sources of information can be used to evaluate the 
relative magnitudes of the variances J,. 

The remainder of this paper will focus on auxiliary 
information provided by relationships between the overall 
stratum-level variances V, and associated within-PSU 
variances. Recall from Wolter (1985, page 41) the 
decomposition, 


Var, = Pale (1.5) 
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where V,, = Var{>/,(n, p,;) Y;;} is the between-PSU vari- 
ance, Vy,=>)4(n,P,;) ‘65, is the within-PSU variance, 
Y,, =E(Y,;|PSU i, stratum h) and o3,,=Var(¥,,|PSU i, 
stratum h). In addition, define Vw = L'D4_,Vy,. 
Estimators of V,, can provide useful auxiliary 
information on the relative magnitudes of V, for two 
reasons. First, for designs with a small n, and relatively 
large n,,, the within-PSU variance estimators V,,, may be 
considerably more stable than V,. Second, in some 
applications (e.g., some of the examples presented in 
Section 5 below), observed variance estimates are consistent 
with a model under which V, is proportional to V,,, ie., 

V, = BY, forall h = 1, ..., L, (1.6) 
where f£, is a fixed constant. The proportionality 
relationship (1.6) would arise if both V,, and V,, are 
proportional to a common scale factor, e.g., (Y,)* for some 
power a. Under relationship (1.6), expression (1.2) may be 
rewritten, 


L =I ji 2 
d= {de ~ tae {2% | ala 1) 
h=l h=l 
Consequently, given a set of stable within-PSU variance 
estimators Le and associated variance-of-variance- 
estimators Var( Vy, ), 


dy. = py (n, - "V2, Vat Pn dl (Xk, Ve ) (1.8) 


is an alternative estimator of d. 

Section 2 considers some of the properties of dys. 
Section 3.1 uses errors-in-variables tests to check the 
adequacy of the proportionality condition (1.6). Section 3.2 
presents two related diagnostics for the relationship between 
V, and auxiliary variables, and for the magnitude of the 
error in the observed auxiliary variables V,,, . 

A simulation study in Section 4 explores conditions 
under which the proposed new estimator an may perform 
better than d_.. This assessment considers both the 
estimation of d as such, and the performance of confidence 
intervals for Y. Section 5 applies the proposed estimator to 
four variables from NHANES III, with emphasis on cases 
for which differences between the proposed estimators ae 
and d_, have a substantial practical effect on assessment of 
the stability of the variance estimator V(Y). Section 6 
reviews the methods developed in this paper and considers 
some possible extensions. 
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2. An estimator based on auxiliary information 


2.1 A within-PSU variance estimator 


A simple estimator of V,,,_ is 


Voy. = mi? > pi? 8, 2.1) 
i=] 

where 65, = my; (Mi; — 1) D7 (Pay Yay — Yai)? Note that 
63, iS approximately unbiased for 63, under a with- 
replacement sampling design within PSU i in stratum h; 
or under simple random sampling without replacement and 
with a small sampling fraction, f,,=N,, n,,. Standard sam- 
pling theory shows that V,,, is approximately unbiased for 
V,, Then an approximately unbiased estimator of 
Var(Vy,) is 


Vac.) nem, = Myaey Vc Vp, )?iet(2/2) 
reat | 


where V,,, = 1,’ Dj, 65,3 See, €.g., Eltinge and Jang (1996) 
and references cited therein. Note that the overall stratum- 
level variance estimators V, are functions of the sample 
means of p,, sf over PSUs in stratum h. In addition, the 
estimators V,,, are functions of sample variances of the 
Pui¥,j Within the PSU (h, i). Thus, for variables Y for 
which Big Ls are approximately normally distributed 
within stratum h, the estimators V, and V,, are approxi- 


mately independent. 


2.2 Properties of dy. 


In the remainder of this paper, the estimator dy. defined 
in expression (1.8) will use Var(Vy, ) as defined in 
expression (2.2). Also, the remainder of this paper will use 
several asymptotic results. These results will use the 
condition that the number of strata, L, is increasing, while 
stratum-level PSU and SSU sample sizes n, and m, are 
allowed to remain small. This is in keeping with many 
practical multi-stage designs that use n, = 2 and moderate 
values of m,. See, e.g., Krewski and Rao (1981) for a 
detailed development of large- asymptotic results. The 
proof of Result 2.1 is routine and is thus omitted. 


Result 2.1. Assume that E(V;,) = O(1) for r = 1, 2, 3, 4 
and define 


and (2.3) 


= 


L See ae A 
J pee L'>\(n, - I) Vy, — VarVyy,)}- 
h=l 
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Then V, and hye are consistent estimators of V,, and 
Sans HEY, ae Bat hae In addition, L” lds isa 
consistent estimator of L™'dyys. 

Section 1 suggested that in some cases, the auxiliary-data 
based estimator d,, might be more stable than the modified 
Satterthwaite estimator d,.. To examine this idea, we will 
compare the variances of d,, and d,. under condition 
(C.1) and the following additional assumptions. 


(C.2) For h =1, 2, ...,L, Vj,(m, —IV,y, are distri- 
buted as independent chi-square random vari- 
ables with m, —1 degrees of freedom, respect- 
tively, where m, is the number of SSUs in 
stratum h; and are mutually independent of V,,. 


(G33) sForallth=1, 2) \-.9L5 1, = 23,and mm, =" for 
some fixed positive integer m, 2 2. 


Arguments similar to those for condition (C.1) indicate that 
condition (C.2) may be satisfied approximately if within a 
given PSU (hf, i), the m, random variables jane are 
approximately independent and identically distributed 
normal random variables. Condition (C.3) restricts attention 
to the common case n, = 2. In addition, condition (C.3) 
requires that an equal number, m,, of secondary units be 
selected within each selected PSU. This _ allows 
simplification of the resulting approximations for the 
variances of dy, as presented in Result 2.2. 


Result 2.2. Assume conditions (C.1), (C. 2), (C. 3), and (1.6), 
and define a = = 4b, He. = Vat(A,)\.b = Au, Up »Cov(A,, By): 
and c= 7 Ms, ‘Var(B,), where A, = L' a Vins B,= 


EF Sys ee — VarVyy,)}: LaDy and epee ba ue en 
Then 


(i) the variances of the Leadine, terms in Taylor expansions 
of L''(dy, — d) and L' (d ms ~ A) are, respectively, 


Vig a0 € 


and 


2 
if OL 
ye mig Ver es 
Lm (se ("m i 


(ui) forall m)2lim, ,,, g(a,,c), lim, ,,.V,,,2 lim, ,..V iy 
where 


An 1) . 
3 (WH £2) 


g(a, b, c) = {23a—-3b+4c)}"! 


{1lc+/144a?+144b7+153c?—288ab+216ac—216bc }. 


Statistics Canada, Catalogue No. 12-001-X 


(iii) for m, 210, lim, ,..V,,, 2lim,_,..Vpy regardless of 
the values of the partune moments lim, ,..(M45 be,» 
LD" Dh Vin LCi Vin)- 


Result 2.2 indicates that for large L, diy may be 
preferable to as provided: (1) the proportionality 
condition (1.6) is satisfied; and (2) the secondary unit 
sample size m, exceeds the lower bound given by 
g(a,b,c) (thus ensuring relatively small variances of the 
V,,,)- This motivates the use of within-PSU variances to 
assess the stability of survey variance estimators, especially 
under sample designs with small numbers of PSUs per 
stratum. For some additional discussion of this point, and 
some specific diagnostics to check the stability of V,,, see 
Eltinge and Jang (1996) and references cited therein. For the 
four cases considered in Table 1.1 and studied further in 
Section 4 below, g(a,b,c) is equal to 4.7, 4.3, 4.6, and 4.8 
respectively, while the NHANES III application had the 
mean of the m, values approximately equal to 22. In 
addition, we are treating V,, values as fixed, and Result 2.2 
depends on the limiting moments of these V,, terms. 
Suppose that V,,/V, had the same moments as F’/ /, 
where F' follows a chi-square distribution on f degrees of 
freedom. Then f = © corresponds to the case in which 
Vy, =Vy for all h, which corresponds to the case in 
which the true d in (1.1) equals the customary value of 
n—-L. 


3. Testing the proportionality condition 
3.1 An errors-in-variables model for V, and V,, 


Development of the alternative estimator dy. in Section 
1, and evaluation of its properties in Section 2, depended 
heavily on the proportionality condition (1.6). One may test 
the adequacy of this condition through the following steps. 
First, note that condition (1.6) is a special case of the 
following model, 


(C4). Forall: pie? De 
Vi, = Bo +BY + % (3.1) 


where B, andB, are constants, and q, is an 
equation error with mean zero and variance 


O ggh: 


Second, recall that V, and V,, are unknown quantities, for 
which we have the unbiased estimators V, and V,,, 
respectively. Using the errors-in-variables model notation in 


Fuller (1987), define the estimation errors 


ES VAAN? Oh atid ey ye SD 
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Under conditions (C.1) and (C.2), the vector (e,,u,)’ is 
distributed with a mean vector equal to (0,0)' and a 
varlance-covariance matrix equal to diag(o,,,,6,,,,) Where 
Seen = (M, —1)'2V, and o,,,, = (m, —1)'2V2,. Under 
AS additional condition (C.3), these variance terms simplify 
to 6,,, = 2V, and o,,, = (m any po 

Expressions (3.1) and (3.2) define an errors-in-variables 
regression model with heterogeneous measurement error 
variances and non-normal errors. In addition Var(V,, ) 
defined in expression (2.2) is an unbiased estimator of 6,,,,, 
and thus provides identifying information for the parameters 
B,,B,; and o,,, in model (3.1) — (3.2). A direct application 
of Fuller (1987, pages 187-189) with equal weights then 
gives the consistent estimators (for increasing L), 


L TL 
B, = pz ome Vey aa ml Vy, =f Vy Vi, (3.3) 
h=l 


Si = 


and 


L 
oF = imax fo, Ln, -1)" 


h=| 


{(L-2)' LO, -By - BVP)? 
= (6.04 + Bi Sun) | (3.4) 
where 
Bs jit. carte aes a> - A 
Ce eV an ye E Seen 
and (3.5) 


6, = 2041) V7 


from condition (C.1). In addition, direct application of Fuller 
(1987, page 188) leads to variance estimators V(,) and 
VB, ), say; details are available from the authors. 


3.2 Two related diagnostics 


In keeping with condition (C.4), the proposed estimator 
d,. is intended for cases in which the V,,, provide useful 
auxiliary information on the relative magnitudes of the 
overall stratum-level variances V,. To identify such cases, 
one simple diagnostic is the ratio {V(V,)}"' {BV (Vy) 4 

Sygnt> ie, the ratio of estimators of the variances of the 
approximate distributions of V,—V, and BV, —V,, 
respectively, under model (3.1)— (3.2). If this ratio is 
substantially less than unity, then use of oboe may be 
indicated. 
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In addition, the performance of the estimator des 
depends heavily on the magnitude of G,, relative to the 


variability of the true within-PSU variances V,,. Define an 
estimator of the reliability ratio (Fuller 1987, page 3) 


conn Poe V, to | [Ze Pe'-a J 


The values of K,, are between 0 and 1; and values of 

x close to unity indicate relatively small errors in the 
estimation of within-PSU variances. Conversely, small 
values of K,, (eg., K,. < 0.7) may indicate that the 
methods of Sections 3.1 — 3.2 may not perform well, due to 
the relatively large sampling errors in the auxiliary 
information V,,,. The numerical work in Sections 4 and 5 
below will consider these diagnostics further. 

The work in this section is based on the assumption that 
Og, > 9. One may develop related diagnostics applicable 
to the case of no equation errors, i.e., Coq. = 0; details are 
available from the authors. 


4. A simulation study 
4.1 Design of the study 


We now use a simulation study to evaluate the properties 
of our degrees-of-freedom estimators, and related variates, 
under moderate-sample-size conditions. We set up the 
simulation procedure as follows. 

We considered four sets of V, values from the NHANES 
III example for the Mexican-American subpopulation 
introduced in Section 1.1. Those four sets of V, are the 
estimated V, values from the variables BMPWT, HAR3, 
TCRESULT and HDRESULT, respectively, and are listed 
in Table 4.1. For each case, we used (8), B,) = (0, 1) and 
9 0, in keeping with the results of Section 3, and thus 
Ve0=-),.-Then; for each: <h-=-lyy DP we “Obtained 
10,000 realizations of the initial estimators (Y,., Yio, Vm» 
Vin.) by assuming that the Y,, are distributed as a normal 
random variable with mean zero and variance 2"V,; that 
V,\(m,, —1)V,y, is distributed as a chi-square random 
variable with m,, — 1 degrees of freedom, where m,; = 11 
for all h and i; and the Y,, and V,,, are mutually 
independent. Note that in our data from NHANES III, the 
average number of secondary units for each PSU i in 
stratum hf is about 11. For each Spray we computed 
Vy, = ba 055 a Ypy and Vy, = 2H iy * Vie and then 
carried out an errors-in-variables regression of V, on Vy, 
with measurement error variance 6,,, = = Var(V,,) using 
formula (2.2). This produced the F etnnient estimators 
(8,,8,), and the degrees-of-freedom estimators oe 
and gue, 
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Table 4.1 
“True” variances V, used in simulation studies 
Stratum Case 1 Case 2 Case 3 Case 4 

l 0.00E+00 0.00E+00 0.00E+00 0.00E+00 
2 0.00E+00 0.00E+00 0.00E+00 0.00E+00 
3 1.56E-04 7.67E-05 1.45E-02 1.76E-02 
4 2.01E-04 3.57E-06 5.60E-02 4. SSE-03 
a 2.82E-04 4.88E-07 1.54E-03 2.91E-03 
6 4.36E-04 0.00E+00 3.73E-03 8.60E-04 
7 7.30E-04 2.14E-06 1.69E-02 1.13E-05 
8 8.80E-04 1.30E-05 2.72E-02 1.40E-03 
9 1.65E-03 1.16E-06 9.24E-03 1.35E-04 
10 1.70E-03 9.46E-07 2.24E-03 1.77E-03 
11 2.73E-03 0.00E+00 2.54E-04 1.32E-03 
12 2.91E-03 5.40E-06 2.75E-02 6.40E-03 
13 4.95E-03 3.73E-07 1.15E-02 5.38E-03 
14 7.25E-03 2.90E-04 3.75E-02 6.97E-02 
15 9.06E-03 9.81E-05 3.46E-01 7.58E-01 
16 1.14E-02 7.47E-06 1.54E-02 4.75E-03 
i 2.69E-02 9.65E-05 7.99E-02 1.01E-03 
18 4.00E-02 1.12E-04 1.44E-01 1.77E-01 
1g 4.27E-02 2.68E-06 8.59E-02 3.88E-02 
20 6.0SE-02 7.57E-06 2.68E+00 7.18E-02 
21 6.45E-02 1.17E-04 1.65E-01 4.52E-04 
22 1.08E-01 1.05E-04 5.41E-01 1.98E-03 


4.2 Coverage rates of t-based confidence intervals 


For the four specified cases, Table 4.2 presents the 
simulated non-coverage probabilities obtained for ¢ -based 
confidence intervals for the population mean Y that used 
the corresponding d. For the severely heterogeneous cases 
(Cases 3 and 4), none of the degrees of freedom measures 
(not even the true d) leads to confidence intervals with 
coverage rates meeting the nominal rates 1 — a. That is, in 
extreme cases, the general Satterthwaite approach can be 
problematic for construction of confidence intervals, 
regardless of whether d, d,,., or dy, is used to determine 
the ¢ multiplier. 

For Cases 1 and 2, the V, values display less severe 
heterogeneity than in Cases 3 and 4. Table 4.2 shows that 
the simulated coverage probabilities with the true d_ for 
these two cases are slightly above 0.95. This overcoverage 
may be attributable to the fact that the variance estimator 
V(Y) is not distributed exactly as a multiple of a 12 
random variable, due to the heterogeneity of the V,. Use of 
the standard degrees-of-freedom term n-—L or the 
modified estimator de. produces confidence intervals with 
coverage rates below the nominal level of 95%. On the 
other hand, use of our auxiliary-data-based term A gives 
simulation based coverage rates close to the nominal 0.95 
level. 

Tables 4.3a and 4.3b display the empirical distributions of 


d and 2t, for the estimators die and des . The simulated 
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standard deviation of ¢, is smaller than that of ¢ dg’ In 
addition, the mean and peda of t, are slightly larger than 
those of ¢, . This is consistent it the undercoverage of 
the fntervals based on ¢, . Thus, under conditions similar to 
those for Cases 1 it 2 (or under conditions with less 
heterogeneity of V,), it is worthwhile to consider the use of 
dy. as a degrees-of-freedom estimator. 


5. Application to a health survey 


5.1 Preliminary model checks 


We applied our proposed methods to the NHANES III 
data described in Section 1. It is important to check the 
modeling assumptions before we apply the proposed 
stability measures. First, for the Mexican-American sub- 
population described in Section 1, Table 5.1 gives values of 
K,,. for the four variables which all have &,, values greater 
than 0.7. 

Second, Figure 5.1 displays the scatter plots of V, 
against V,,, for the four variables with equal scales used for 
the horizontal and vertical axes. It shows that a linear 
relationship for the corresponding variables is plausible 
even if the relation would not be perfect and there are some 
outliers. Consequently, those four variables might be appro- 
priate for the auxiliary-data-based method developed in 
Sections 2 and 3. 
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Table 4.2 
Observed non-coverage rates for nominal 95% confidence intervals with V, = V,, in simulation study 
Case 1 Case 2 Case 3 Case 4 
True ds 6.26 6.04 2.38 2.20 
Non-Coverage with ¢ 0.0428 0.0443 0.0162 0.0164 
Non-Coverage with #,_) 0.0744 0.0788 0.1220 0.1263 
Non-Coverage with 1, 0.0552 0.0567 0.0911 0.0905 
mS 
Non-Coverage with ¢, 0.0428 0.0466 0.0224 0.0220 
LAY 
Table 4.3a 7 " 
Means and quantiles of degrees-of-freedom estimators d,,,; and dy,: Cases 1 and 2 
Cases True d Est. ‘Mean d SD(d ) 7Q(0.05) Q(0.25) Q(0.50) Q(0.75) Q(0.95) 
1 6.26 ane 9.33 3233 4.45 6.86 9.01 11.41 15.30 
ann 6.52 0.82 5.06 5.99 6.57 7.10 7.78 
2 6.04 ae. 8.87 Z95 4.35 6.69 8.72 10.97 13.99 
dys 6.34 0.96 4.67 5.69 6.42 7.06 7.80 
1 Mean denotes the average of the estimates, taken across all 10,000 replications. 
2 Q() indicates the quantile of the estimator, taken across all 10,000 replications. 
Table 4.3b 
Simulated non-coverage probabilities; and means and quantiles of t-multipliers for nominal 95% confidence intervals: 
Unequal true variances, cases 1 and 2 
Cases Est. os eae >M(2t,) SD(2t,) 3Q(0.05) Q(0.25) Q(0.50) Q(0.75) Q(0.95) 
] (he 0.0552 4.62 0.36 4.26 4.38 4.52 4.75 ape) 
das 0.0428 4.83 0.16 4.64 4.72 4.80 4.90 ets) 
n-L 0.0744 4.15 
True ds 0.0428 4.85 
2 a, 0.0567 4.66 0.36 4.29 4.4] 4.55 4.78 5.41 
dye 0.0466 4.87 0.21 4.64 4.72 4.83 4.97 5.28 
n-L 0.0788 4.15 
True ds 0.0443 4.89 


' 1-@ is the simulated non-coverage probability of confidence intervals computed using estimated d.f.’s 
? M(2t,o7s) is the average of twice of the 97.5% #-percentile value 


3 Q(.) indicates the quantile of 2¢ 


Table 5.1 


0.975, d” 


taken across all replications. 


Kx.» estimates of model parameters, model diagnostics, and degrees of freedom estimates for four NHANES III variables 
(Mexican-American (Age 20-29) subgroup) 


Variables ,, 


BMPWT 0.75 
HAR3 0.75 
TCRESULT 0.88 
HDRESULT _ 0.90 


B, se( B,) 
-0.0013 0.0039 
-0.000009 0.000012 
-0.146 0.0493 
-0.042 0.0098 


B, 


1gh35 
1.095 
2019 
6.650 


se(B, ) Si 


0.5429 
0.3991 
0.6252 
0.9988 


mulation based 
p-value for 
Hy: By = 0 
0.3815 
0.4229 
0.0606 
<0.0001 


Simulation based 
p-value for 
Hy: By =1 

0.3541 
0.3400 
0.2259 
0.1506 


S qq. 


-0.000 
0.000 
-0.178 
-0.017 


-0.43 
-0.83 
-0.77 
-0.91 


15.49 
14.94 
5.88 
5.45 


10.04 
8.30 
6.59 
593 
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Variable = BMPWT 
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0.00 tng T “Sa a 
0.00 0.02 0.04 0.06 0,08 0.10 0.12 
Vin 
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s 
DG 
2.0 4 
V, 15 4 


0.0 


1.0 1.5 2.5 3.0 
Vn 
Figure 5.1 


5.2 Anad hoc test of G,, = 0 under condition (C.1) 


For all four variables considered in Table 5.1, the direct 
estimates G qq Of equation error variance (3.4) were negative 
or close to zero. That suggests that our x7 -based estimator 
of o,,, as given in Section 3.1 might be too conservative or 
that G,, is indeed close to zero. This suggests that we need 
to re-examine the distributional assumption (C.1) in the 
NHANES III example. To do this, we considered the 
simulated distribution of 7,, = G aq! Sees Where division by 
6,, 1S used to avoid scale problems. The conditions and 
simulation design were as described in Section 4.1. 

Table 5.2 reports results for 6,, from expression (3.5), 
and & ie computed from expression (3.4) with f, set equal 
to zero and with B,, computed from expression (3.3). Table 
5.2 reports the mean, standard deviation and selected 
quantiles of the simulated distribution of 7, for the four 
variables. Table 5.3 reports the corresponding quantities for 
F,, computed from G aq &iven by expression (3.4) and with 


at : 
B, and B, computed from expression (3.3). 


Statistics Canada, Catalogue No. 12-001-X 


Jang and Eltinge: Use of within-primary-sample-unit variances to assess the stability 


Variable = HAR3 


0.00030 


0.00025 


0.00020 -- 


0.00015 


0.00010 -- 


0.00000 0.00005 0.00010 0.00015 0.00020 0.00025 0.00030 


Vn 


Variable = HDRESULT 


0.6 + 


0.4 : 


0.4 0.8 


Vin 


Plot of V,,, vs. V, for M-A (Age 20-29), Variable = BMPWT 


The results reported in Tables 5.2 and 5.3 lead to an ad 
hoc test of Hy: 6,, = 0. Specifically, if the observed ratio 
F,, falls above the upper 0.95 simulated quantile, then the 
assumption that G,, = 0 may be problematic. Conversely, 
an observed /,, below the .05 simulated quantiles in Tables 
5.2 or 5.3 might indicate that 6,,, is conservative, or may 
indicate violation of other parts of condition (C.1). 

From Table 5.1, the values of /,, for the variables are 
between -0.91 to -0.43. Except for HDRESULT, we do not 
have any strong evidence of violation of the model 
assumptions. However, for HDRESULT, the ratio 7, = 
-0.91 falls between the 0.01 and 0.05 quantiles reported in 
Table 5.2 and 5.3 for case 4. In general, values of 7, that 
fall above the 0.95 or 0.99 quantiles of Tables 5.2 or 5.3 
would be consistent with values of 6,, greater than zero. 
The observed value /,, = -0.91 is not necessarily consistent 
with G,, > 0, but may indicate violation of one or more 
conditions in (C.1)-(C.4). 
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Table 5.2 
Means and quantiles of Fg, = 6269. (By = 9) 
Cases ™( Faq) SD( Faq ) 7Q(0.01) Q(0.05) Q(0.10) Q(0.25) Q(0.50) Q(0.75) Q(0.90) Q(0.95) Q(0.99) 

l -0.50 0.66 SETA ey 1530 -1.15 -0.99—_-0.79 0.16 0.54 0.60 0.65 

2 -0.48 0.68 Laz “132 -1.16 -0.99  -0.76 0.23 0.57 0.62 0.66 

3 -0.19 0.42 -1.01 -0.84 -0.74 -0.53 = -0.20 0.17 0.38 0.46 0.55 

4 -0.20 0.39 -1.00 -0.82 -0.72 -0.51 -0.20 0.11 0.34 0.44 0.56 


_M denotes the average of the estimates, taken across all 10,000 replications. 
* Q(.) indicates the quantile of the estimator, taken across all 10,000 replications. 


Table 5.3 
Means and quantiles of Pgq= ze F gg 
Cases "M(Fqq ) SD( qq) 
1 -0.56 0.62 -1.85 -1.34 
2 -0.56 0.62 -E91 * 8-137 
3 -0.24 0.42 -1.16  -0.90 
4 -0.24 0.38 -1.09 — -0.87 


-1.17 


-1.18 


-0.79 


-0.75 


7Q(0.01) Q(0.05) Q(0.10) Q(0.25) Q(0.50) Q(0.75) Q(0.90) Q(0.95) Q(0.99) 


-1.00 -0.80 0.05 0.38 0.44 0.52 
-1.00 -0.78 0.06 0.35 0.42 0.50 
-0.57 -0.22 0.12 0.29 0.36 0.45 
-0.53 -0.22 0.06 0.25 0.33 0.44 


'M denotes the average of the estimates, taken across all 10,000 replications. 
* Q(.) indicates the quantile of the estimator, taken across all 10,000 replications. 


5.3 Coefficient estimates and degrees-of-freedom 
estimates 


Because our data were consistent with 6,, = 0 for all 
four cases, we used the methods of Fuller (1987, page 124) 
to produce estimates of B, and B, appropriate for a model 
(3.1) -—(3.2) with no equation error; details are available 
from the authors. Table 5.1 also reports the resulting 
coefficient estimates 8, and B,, and their standard errors, 
se(B,) and se(B,). Recall from Section 3.1 that under 
model (3.1)—(3.2), if B,) =0 and fB, #0, then each 
stratum variance V, is a constant multiple of the within- 
PSU variance V,,, and de in (1.8) may be an appropriate 
estimator of d. Section 5.2 already considered the condition 
6,,, = 9. To test the null hypothesis H,: B, = 0, we use 
the test statistic, ¢, = B,/se(B,). In some practical errors- 
in-variables work, quantities like ¢, are compared with a 
standard normal or ¢ reference distribution. However, 
simulation work based on the four cases from Section 4.1 
indicated that the null distribution of {f deviated 
substantially from these customary reference distributions. 
This is due to the very skewed distributions of the response 
variables V, used in the errors-in-variables regression. 
Consequently, we used standard methods to develop a 
simulation-based reference distribution for ¢). Column 7 of 
Table 5.1 reports the resulting left-tailed p-value. (Due to 


negative point estimates ,, we have chosen to report the 
left-tailed p-values here. In other cases, it may be of 
interest to report right-tailed or two-tailed p-values for 
B,). There is strong evidence against H,: 8B, = 0 for the 
variable HDRESULT, and the moderate evidence against 
Hy: By = 9 for TCRESULT. Thus, it may not be appro- 
priate to use d,,, for these two variables. Now consider the 
slope coefficient B,, and suppose that o,,, = 0 so q, = 0 
with probability one. Then expressions (1.5) and (3.1), and 
the nonnegativity of V,, implies that 0 < V,,=V,—V,, = 
B, + (B, — lV,,- Consequently, if B, = 0, then B, 21 
and B, =1 is equivalent to V, = V,,. This final condition 
is of practical interest because some authors have noted 
cases in which V;, is small relative to V,,, or equivalently, 
V, = V,y,. See for example, Wolter (1985, page 46). To test 
H,: B, =1 against the one-sided alternative H,: 8, > 1, 
we used the statistic ¢, = (B, —1)/ se(B,)- For reasons 
similar to those for f,, we developed simulation-based 
reference distributions for ¢, under each of Cases | through 
4. Column 8 of Table 5.1 reports the resulting one-tailed 
p -values. 

The last two columns of Table 5.1 report the degree-of- 
freedom estimators d,,, and d,. For HAR3 and BMPWT, 
d 5 gives substantially larger values than ee 


mM 
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6. Discussion 


This paper has considered estimation of a degrees-of- 
freedom term d used to quantify the variability of a standard 
design based variance estimator V(Y). The fundamental 
issue is that under a design involving heterogeneous 
stratum-level variances and small numbers of primary 
sample units selected per stratum, the Satterthwaite-type 
estimator ake may perform poorly. We developed an 
alternative estimator a based on within-primary-sample 
unit variance estimators V,,,. This alternative estimator is a 
solution to an unbiased estimating equation (1.1) for d, 
provided the proportionality condition (1.6) is satisfied. 
Also, the variance of the approximate distribution of ane is 
smaller than that of d,, provided the number of secondary 
sample units selected within each primary unit is large, in 
the sense defined by Result 2.2. 

Section 3 developed errors-in-variables methods for 
testing the adequacy of the proportionality condition (1.6), 
and suggested some related diagnostics. The simulation 
study in Section 4, in conjunction with the data analysis in 
Section 5, indicated that under moderate amounts of 
heterogeneity, ds can perform better than a, in terms of 
the distributional properties of these estimators of d, and in 
terms of the coverage rates and widths of associated 
confidence intervals for the population totals Y. However, 
as one would expect from standard large-sample theory, 
neither estimator performs well under severe heterogeneity. 

One could in principle consider use of the errors-in- 
variables estimators (Bos B.. Sq.)> in conjunction with the 
V, and V,,, to construct an alternative estimator of d that 
will be consistent under the general errors-in-variables 
model (3.1) - (3.2), and will not require the restrictive 
condition (1.6). However, simulation results in Jang (1996) 
indicated that the resulting estimator d,,,, say, did not 
perform well under the design conditions used in Section 5. 

The principal results of Sections 1 through 3 extend 
readily from the within-primary-unit variances V,,,, to more 
general auxiliary variables X,. For such extensions, the 
principal issues remain the adequacy of the proportionality 
approximation (1.6); and the amount of sampling error in 
the auxiliary estimators Y,, say, relative to the error in the 
basic stratum-level variance estimator V,. 
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Appendix A 
Proof of result 2.2 


Consider a nonlinear function B™' A’ of two estimators 
A and B with means p, and iz, respectively. Then, the 
variance of the leading term of a Taylor expansion of 
BA? 4s 


Ay? w u' 
—4 Var(A) — 4—4Cov(A, B) + —4 Var(B) .(A.1) 


He B Up 


Now we define the following two estimators: L™d,, = 
BA? and L'd,,=B;'42, where A,=L'Y4,V,, B, = 
Lip Sie Vis far Sheers oad ie Dyan — 
Var(V,,,)}- 

Assume conditions (C.1), (C.2) and (C.3). In addition, 
define F 1p and F 1d,, ' be the leading terms of Taylor 
expansions of L'd,, - HSA vie 
respectively. Also, recall that if D is distributed as a chi- 
square random variable on d degrees of freedom, then 
V(D) = 2d, E(D*) = d(d + 2)(d +4), and V(D’) = 
8d(d + 2)(d +3). Then the corresponding components of 
Var(F, a) and Var(F, Fe in (A.1) are 


and L'd,, — HaHa,» 


L 
Nan Aaah, oars 


h=) 
1p 
Var(A,) = 2(m, —1)'L° Vy, 
h=1 
Ib, 
Var(B,),=.96L, ies 
h=1 


Var(B,) = 8(m, — 1)? (m + Wap set 


h=| 


Mb, 
CoviAeBin=l2b oils 


h=1 
and (A.2) 
iE 
Cov(A,,B,) = 4(m) - 1) 'L?Y Vy. 
h=1 


Since we assume n, = 2 and m, =m, for all h = 
12, oe al awe have 


5 
Ae 
| 


MeO E BP 14)" (9B (A.3) 
and 


fig? eee ig Bi (A.4) 
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Under condition (1.6), Wy, = By 4, 


ice SB las: 
Var(A,) = (m - 1)B; Var(A, ), 


Var(B,) = 12(m, + 1) '(m, — 1) Bi Var(B,) 
and 
Cov(A,, B,) = 3(m, — 1)B;Cov(A,, B,) (A.5) 
Substituting (A.5) into (A.1) leads to, 
Var(F 5) 
fis, 
= =(m - ye Var(A, ) 
B, 
3 
- a — 1)Cov(A,, B,) 
B, 
“HER te aly 
lg de ple = Army = 1) 
= 5% l)a 5% b+ 21m, + 2° (A.6) 


where Var(L- ‘dy. i= a—b+c. With large L, Var(F a, = 
(nla = (m, —1)b + {3(m, + 2)}'4(m, -l)’c. Thus for 
large L, ACF Cie pt) = ye 2) a — (my = 2)'b'+ 
{3(m, +2)}- '(4m —11m,—2)c. Therefore, lim 1-9 Ad 
lim, ,.Viw 20 if m)2lim, ,,.{2Ga—3b+4c)}" {11c+ 
V144a’ + 144b? + 153c? — 288ab + 216ac — 216bc}. In 
particular, lim, ,..V,,, — lim,_,..V, becomes greater than 
or equal to zero when m, = 10 regardless of values of a, b, 
and c. Because it is an increasing function in mp, for all 


vawwes of yn. 210. lim). Vin, Se es Vp 
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Semiparametric regression model for complex survey data 


Zilin Wang and David R. Bellhouse ' 


Abstract 


A semiparametric regression model is developed for complex surveys. In this model, the explanatory variables are 
represented separately as a nonparametric part and a parametric linear part. The estimation techniques combine 
nonparametric local polynomial regression estimation and least squares estimation. Asymptotic results such as consistency 
and normality of the estimators of regression coefficients and the regression functions have also been developed. Success of 
the performance of the methods and the properties of estimates have been shown by simulation and empirical examples with 


the Ontario Health Survey 1990. 


Key Words: Complex survey; Domain estimates; Nonparametric regression; Smoothing. 


1. Introduction 


In practice, many surveys are used to explore a relation- 
ship between a response variable and explanatory variables 
and to build predictive models. Hence, it is necessary to 
develop techniques that apply stochastic regression models 
to survey data. Although nonparametric regression tech- 
niques have been widely applied in many fields of statistics, 
not much attention has been paid to them in the field of 
complex surveys due to the complexity of the data structure. 
The correlation induced by clustering and unequal proba- 
bilities of selection of the sample cause survey data to be 
neither independent nor identically distributed. As a result, 
standard nonparametric regression methods are often inap- 
propriate for analyzing sample survey data. 

There is some work, for instance Breidt and Opsomer 
(2000), Montanari and Ranalli (2005), and Zheng and Little 
(2004), on nonparametric regression techniques that have 
been developed for survey data. However, as in the conven- 
tional way of applying regression techniques, most of this 
work uses model-assisted approaches to estimate descriptive 
population quantities and parameters related to the de- 
scriptive quantities. In this paper, we are interested in the 
application of nonparametric regression techniques to 
exploring the relationship between the response variable and 
covariates, as well as prediction using auxiliary information. 
Bellhouse and Stafford (2001) extended a local polynomial 
regression technique to conduct flexible regression model- 
ling for complex survey data. However, their paper dealt 
only with a simple nonparametric regression function. Here 
we extend their enquiry to a case of several independent 
variables, including indicator variables that often appear in 
regression analysis for survey data. 

We consider a partially linear semi-parametric regression 
function defined as E(y|X,z) = XB +G(z), where 
G(-) is an arbitrary function and B is an unknown 


p-—dimensional parameter vector. In this semi-parametric 
regression model, the explanatory variables are represented 
separately in two parts: a nonparametric part and a 
parametric linear part. It is of interest to estimate both the 
functional form of the nonparametric part of the model and 
the parameters that are included in the parametric part of the 
model. We put the categorical explanatory variables and 
continuous variables with assumed linear dependence in the 
parametric part of the model, XB, and a variable with little 
information on the functional form in the nonparametric part 
of the model, G(z). This partial linear semi-parametric 
model not only has a priori motivation as a data analytic tool 
and retains an important interpretive feature, it also eases the 
high dimensional problem created by factors and some 
covariates by including them in the parametric part of the 
model. 

A similar model has been developed for independently 
and identically distributed data independently by Robinson 
(1988) and Speckman (1988). In these papers, the esti- 
mation is conducted in three steps. In the first step, the 
means of the response variable and the parametric 
independent variables, conditional on the nonparametric 
variable, are treated as a function of that variable and 
smoothed; in the second step, the linear coefficients are 
estimated by regressing the residuals from the smoothed 
response variable on the residuals from the smoothed 
parametric covariates; finally, the difference between the 
response variable and its prediction from the regression 
model is smoothed in a similar manner to provide an 
estimate of the nonparametric part of the regression 
function. It has been shown in Robinson (1988) and 
Speckman (1988) that the resulting estimators are root-n 
consistent when the model is correct and the data points are 
independent and identically distributed. The objective of our 
paper is to apply this smoothing technique to survey data 
while allowing for a complex sampling scheme. 


1. Zilin Wang, Department of Mathematics, Wilfrid Laurier University, Waterloo, ON, Canada, N2L 3CS. E-mail: zwang@wlu.ca; David R. Bellhouse, 
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We use the local polynomial regression estimation 
technique developed in Bellhouse and Stafford (2001) to 
conduct all the smoothing during the estimation process. A 
key element in accomplishing the local polynomial regres- 
sion technique from Bellhouse and Stafford (2001) is 
binning, which follows the work of Bellhouse and Stafford 
(1999) in density estimation. In many survey data sets, a 
continuous variable may be naturally binned; for example 
age may be recorded as age last birthday. In general, bins 
correspond to the disjoint sets of values of a continuous 
covariate, and thus can be regarded as domains. At the level 
of the sample, we estimate the domain mean of the variable 
of interest by dividing the weighted sum of the variable 
within the domain by the sum of the weights within the 
domain. In Bellhouse and Stafford (2001), the response 
variable is binned according to the values of the covariate, 
and discretized, and the domain means of the response 
variable are smoothed to obtain the regression function. 
When the sample size is large and the number of bins is 
relatively small, then estimators based on binning are 
functions of domain estimators whose inferential properties 
can be readily derived from results in Shao (1996) and 
Serfling (1980). One of the practical advantages to binning 
is that it can reveal information on an obscured trend in a 
complex survey, which is sometimes quite important when 
the scale of the complex survey data set is large. There are, 
usually, multiple observations at each set of covariate values 
in these data sets. 

An example that illustrates these features of binned data 
is taken from the Ontario Health Survey. The survey was 
conducted by Statistics Canada in 1990 with 61,239 
individuals living in Ontario, Canada. The data were 
obtained by a stratified two-stage clustered design. The 
strata were the urban and rural areas covered by each of the 
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public health units in the province of Ontario. Within each 
stratum enumeration areas were randomly selected, as were 
households within each enumeration area. The purpose of 
this survey is to measure the health status of the people of 
Ontario and to collect data relating to the msk factors of 
major causes of mortality and morbidity in Ontario. In this 
example, we examine people’s weight as a function of age. 
In the Ontario Health Survey, age was given only to age last 
birthday. The measurement we use for a proxy of weight 
here is called body mass index (BMI) which is calculated as 
weight in kilograms divided by the square of height in 
meters. BMI is used as one of the indicator of a person’s 
obesity level. Normally, a person with a BMI below 18 is 
considered underweight and a BMI greater than 30 suggests 
obesity. BMI is used as an appropriate measure only for all 
persons between the ages of 18 and 64 with the exception of 
pregnant and breast feeding women. Consequently the 
sample size is reduced to 44,457 eligible respondents that 
have 47 distinct possible ages or bins. 

In the left panel of Figure 1, the age trend of body mass 
index is plotted. It is readily seen that the “black cloud"-like 
scatterplot masks the relationship between age and body 
mass index. Now, if we calculate mean of the body mass 
index at each distinct point of age, and plot the binned mean 
estimates of the body mass index versus age, we can obtain 
the plot in the nght panel of Figure 1. It is obvious that a 
binned mean provides more visual information than the raw 
data does. Large-scale data sets not only can result in non- 
informative plots, they also make the estimation process 
computationally very cumbersome. Hence, it is natural in 
complex survey data analysis to bin the data into domains 
according to distinct values of a discretized covariate. 
Further, estimators from binning are functions of domain 
estimators. 
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Figure 1 Comparison of the scatter plots of the binned and unbinned data from the Ontario health survey 
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One drawback to binning is that number of bins cannot 
grow asymptotically with the population if the data are 
naturally binned, as with the age variable in the above 
example. In such a case, the population level nonparametric 
estimators will remain biased as estimators of super- 
population functions due to a fixed bin size. In our 
framework, we assume that the bins induced by the distinct 
values of the covariate are the same in the population as in 
the sample; similarly in smoothing we will take the 
bandwidth to be the same at the population level as at the 
sampling level. We will show that the sample estimators are 
design consistent estimators of the corresponding finite 
population parameters and functions, though not of their 
superpopulation counterparts. In the Ontario Health Survey 
data example, the same set of distinct ages appears in both 
the population and the sample. 

The paper is organized as follows. Superpopulation 
working models leading to the estimation procedures in 
survey data are introduced in Section 2. In Section 3, we 
derive all the moments of the estimates obtained and 
establish some asymptotic results. A simulation study and 
an empirical illustration of the estimation method carried out 
using the 1990 Ontario Health Survey (1992) appear in 
Section 4 and Section 5. Section 6 concludes with a 
discussion of assumptions made and some future work. The 
Proofs of all lemmas and theorems in Section 3 are given in 
an appendix. 


2. Semiparametric regression model 
and its estimation 


We take a typical approach to complex survey data 
analysis. First, we assume a working model on the finite 
population under the assumption of independent obser- 
vations. Model parameter estimates then become the finite 
population parameters, or census parameters, to be esti- 
mated from the survey sample. Once the finite population 
target parameters have been defined we assume a more 
realistic model on the finite population in order to obtain 
inferences about these parameters. This is done in the next 
section. Consider a finite population of size N with a vector 
of measurements (y,, X;,,Z,) attached to unit k, k = 
1,..., N, where y, represents an observation of the 
response variable and (x,, z,) represents a vector of 
observations of the explanatory variables with length 
p +1. As a working model we imagine that the response 
variable is generated by the following partial linear 
regression model, 


Y = G(z)+ XPt+e (1) 


where Y is the vector of responses and € has entries that 
are independent and identically distributed with mean zero 
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and constant variance. The function G(-) is an arbitrary 
function of z and B is an unknown _p -dimensional 
parameter vector. The N x p matrix X corresponds to the 
linear part of the model and contains either continuous or 
discrete explanatory variables which are random. The term 
G(z) is the nonparametric part of the model. We assume 
that z is non-stochastic and measured on a continuous 
scale, discretized into D distinct values. Additionally, it is 
imagined that E(e|z, X)=0. There is no interaction 
between X and z in the model. 

We are interested in estimating population level versions 
of G(-) and the parameters B. We first develop expressions 
for these, guided by the estimation procedures in Robinson 
(1988) and Speckman (1988). In particular, we begin by 
taking the expectation of both sides of (1) conditional on z: 


E(¥|z) = E(X|z)B + G(z). (2) 
Then we subtract (2) from (1) to obtain 
Y - E(Y|z) =(X- £(X|z))B te. (3) 


To define the population version of B in (3), we will replace 
E(Y¥|z) and E(X|z) in (3) by their population level 
estimates and estimate B by the method of least squares. 

For the population level estimates of E(Y|z) and 
E(X|z), we adopt the local polynomial smoother in Jones 
(1989), in which binning is an essential part of the 
operation. Let the discretized Z variable take values 
21, ype let the vectors of means in the,bins Of z,,..., Zp 
be Y= (i ah Y;) and X, = ee! wl: ue,,) for 
j =1,..., p, respectively. Also, let P, be the population 
proportion of observations in the d" bin for d = 1, ..., D. 
Then denote the population smoothed conditional ex- 
pectations of Y and X, at the point z, by m,(z,) and 
m,(z,), respectively. Given that K(-) is a kernel function 
satisfying [K(t)dt =1 and JK(t)’dt < © and h is the 
bandwidth and using the principle of local polynomial 
regression technique, we minimize 


= P, > ' ’ qy2 
dain 
Laney 
x K{ h ) (4) 

and 

2 P, = ' ’ qGix2 
D5 {Xa Sol timid Ger ae Meces Yq (Za = 2y)°} 

a= 


Zz, -—2Z 
x K|-4 d 
[274] 
with respect to a ’s and y’s so that the population esti- 


mated (smoothed) conditional expectations of y and X, 
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on z,,m,(z,) and m,(z,), are the solutions of a, and y, 
for equations (4) and (5). Specifically, 


MZ, = 6 (ZL Kyl) 2 Ky X, 
and 
m,(z,) = e' (Z'’KyZ)'Z'KyY 


where gq is the degree of the polynomial smoother, e is a 
(q +1)x1 vector in the form of (1, 0, 0,...,0)", and Z 
and Ky, are respectively defined as 


ign (Zc, 
LZ 2 Co ay 

Z= (6) 
l intpdlis Sea imads 


and Ky, =diag (PK ((z, —z,)/h), ..., P>K((zp —2,)/h)) /h. 

With the census estimators of the conditional expecta- 
tions m,(z,) and m,(z,), we define a N x p matrix My 
anda N x1 vector M, as, 


m,(z,) m,(z,) m,, (2Z;) 
m (Zz) om(z7) m,(Z;) 
M, = 
Wt (Zi) ally (aay) m,(Zp) 
m (Zp) m,(Zp) m,(Zp) 
and (7) 
m, (Z;) 
m, (Z,) 
M, = 
m, (Zp) 
m, (Zp) 


Note that the d' blocks of My and M, are of the 
dimensions of N, x p and N, x1, respectively, where 
N,, be number of observations that fall in the d" bin and 
XN, = N. Replacing the conditional expectation matrix, 
E(X | z), and vector, E(Y | z), in (3) with their estimates, 
M, and M,, and using the general estimating equations 
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framework suggested by Godambe and Thompson (1986) 
for the least squares estimation, we can obtain the finite 
population versions parameters (census estimators) of B, 
namely B, by solving 


N 
u(@) = > (x, - My)’ (y%, - My) 
k=1 


N 
= yr 3 Mx) (x; —M,,)B 
k=l 
=o (8) 


where M,, is the k™ row of the N x p matrix M, and 
M,, isthe k" element of the N x1 vector M,. The finite 
population parameter vector 0’ is composed of (B’, 
m,(z), my (z)’), where m,(z) is a vector of the form 
(m,(z)’,...,m, (z)’) with m_,(z) = (m,(z,), -.-, m;(Zp)) 
for j =1,..., p and m,(z) = (m, (z,),..., my (Zp)). Hence, 
the closed form expression for the estimator (census 
parameter) B is 


B = ((X — Mx)’ (X — Mx)) '(X — Mx)" (¥ — My). 


Once B is obtained, the difference between the response 
variable Y and the product XB is treated as the dependent 
random variable and the function G(-) is estimated in 
accordance with the following model 


Y —-XB=G(z)+e. 


The finite population version of G(z) at z,, namely 
@(z;7); is 
g(z,) =e’ (Z’KyZ) 'Z’Ky(¥ — XB), 


where X isa D x p matrix of the form (X,,..., X,). 

Realistically, we cannot access the whole population. 
Instead, we can only observe a sample drawn from the 
population using a certain probability sampling design. Let 
s be the set of m sample units with sample (),, X,, Z,, %;) 
for k es, where w, is the sampling weight for unit k. 
Additionally, we assume that there is complete response so 
that the inclusion probability is equal to the reciprocal of the 
sampling weight. We assume further that the bins induced 
by the distinct values of z are preserved from the 
population to the sample. This is appropriate in a variable 
such as age recorded to age last birthday. 

Using the local polynomial regression technique for 
complex survey data in Bellhouse and Stafford (2001), we 
use the sampling versions of the objective functions in (4) 
and (5) as follows, 

D 


ip pe 
SY =4{7, - A — 0,(z, 
d'=)| h 


gol) Need bd Wire fe bd 


x K{ = a (9) 
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and 


D A 
Pa 
dn 


al Ca: fomea Dk a 


. x(# =a (10) 


where y and x, are sample estimators of Y and X, and 
are of the forms (j,, ..., yp) and Cees, xe)h 
respectively, and p, is the weighted sample proportion of 
observations in bin d. Consequently, we have the survey 
estimator of m,(z) and m,(z) at z,, given by 


ssid Sanh 5 CF Dace 


mh (zq) =e" (Z’KyZ)'Z'Kyx, 

and 2 FELT) 
wh, (zy) = e' (Z’KyZ)'Z' Kwy, 

where Z has the same form as in (6) and K., is defined as 


Kaw “diag (5, K(G eile)! 10), ©. SDs (Zintearean) a): 

We can also construct the n x p matrix M, and nx 1 
vector M, using the same method that we used to construct 
M, and M, in equations (7). That is, we use sampling 
estimators m,(z,) and m,(z,) that are shown in (11) to 
obtain 


m,, (z,) my, (z,) car Mm, (z,) 
m, (Z,) My, (Z,) was Mr, (Z,) 
Nae 
mh, (Zp) My, (Zp) <M, (Zp) 
m,, (Zp) My, (Zp) My (Zp) 
and 
mM, (z,) 
mM, (z,) 
Vives 
(Zp) 
my (Zp) 
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Let n, be the number of observations in the d" bin such 

that Xn, =n. Similar to My and M, in (7), the di 

blocks of M, and M, are e the dimensions of n, x p 

and n, x 1, fe eeaively. 

Analogous to the population estimating equation (8), the 
sampling estimating equation for B is 

a(6) = )(x, -M 

kes 


-> (x, -M 


kes 


= 0, (12) 


where 67 = (B’, m,(z), m, (z)') is the sampling esti- 
mator of 0’ = (B’, m,(Z), m, (z)’). Note that a similar 
approach was considered by Fuller (1975) and Binder 
(1983). Nevertheless, the solution to (12) provides the 
closed form of B as 


PCy i M,)™, 


xk is (x, - My ) Bw, 


B = ((x—M,)’ W, (x-M))'(x-Mx)’ W, (y—M,), 


where W, is an nxn weight matrix with design weights 
w, on the diagonal entry for k es, y is an n x1 vector 
containing the sample observations of the response variable 
and x is an nx p matnx consisting of the sample 
observations of the covariates. 

Using the sample estimates of B and denoting x as a 
D x p matrix of the form (%,, ..., X,), we can obtain the 
sampling estimate of g(z,) as 


&(z,) = e'(Z’KyZ) ‘ZK, (¥ — XB). 


Again, if q and h are the same as for m,(z,), the 
expression for ¢(z,) simplifies. 

When applying local polynomial regression techniques to 
obtain the estimators of conditional expectations as well as 
the arbitrary function G(-), we need to choose an ap- 
propriate bandwidth h. Because binning is involved in all 
aspects of the estimation process and since we assume that 
bins induced by the distinct values of z are preserved from 
the population to the sample, we argue that the same 
bandwidth should be used for obtaining both the census 
estimators and the sample estimators. Since we do not have 
all the observations of the finite population, we use the 
sample to choose the appropriate band width. In this paper, 
we adopt the method in Fan and Gijbels (1995), where the 
authors developed a data-driven bandwidth selector that 
combines the ideas of the plug-in and the cross-validation 
methods for the identically and independently distributed 
data. When applying this data-driven method to our case, 
criteria, such as the residual sum of squares and mean 
square error, of the resulting estimates of the conditional ex- 
pectations are needed. By noting that those criteria depend 
on the estimated conditional expectations or regression 
functions and the derivatives of the regression functions, we 


Statistics Canada, Catalogue No. 12-001-X 


252 Wang and Bellhouse: Semiparametric regression model for complex survey data 


can use the objective functions defined in (9) and (10) to 
obtain not only the survey estimates of regression functions, 
but also the derivatives of the regression functions. For more 
details, see Wang (2004). 


3. Design properties of sampling estimators 


3.1 Notation and assumptions 


In showing design properties of the estimators, we follow 
Samdal, Swensson and Wretman (1992) and Isaki and 
Fuller (1982) in considering a nested sequence of popula- 
tions (WOvfor ve= 1} 2, 14 )\such thay, EUs CU sexe 
All population quantities, sample sizes and values, and 
survey estimators are indexed by v. However, for ease of 
notation we drop v as a subscript for these quantities. We 
denote the expectation and variance with respect to 
sampling design as £, and Var,, respectively, and in 
accordance with the above nested populations, we define 
design-based consistency and asymptotic unbiasedness as in 
Thompson (1997, page 167). 

In what follows, the development of the asymptotic 
results for the estimators will depend on the asymptotic 
normality and consistency of the estimates of means and 
totals. We will not restrict ourselves to specific sampling 
designs; instead, we assume that all the survey totals that 
appear in the estimators are of the Horvitz-Thompson type. 
Hence, the consistency and asymptotic normality of 
estimators are subject to the standard regularity conditions 
on the sampling designs for the consistency and normality 
of Horvitz-Thompson type estimators, which have been 
studied by Madow (1948), Hajek (1960), Bickel and 
Freedman (1983), Krewski and Rao (1981) and Shao 
(1996). The aforementioned literature shares some restrict- 
tions on the sampling design. An implication of these 
restrictions is that no survey weight is disproportionately 
large, the total number of first stage sampled clusters or 
primary sampling units is increasing, but with a growing 
gap between sample and population. In addition, a Liapunov 
- type condition ensures that the variables z, x and y 
develop in a regular manner as v tends to infinity. 

We will use the result that any vector of estimators of 
totals from binned data is asymptotically multivariate 
normal, provided that the conditions in the previous 
paragraph are met and the number of domains is fixed. This 
is obtained through application of results in Shao (1996, 
page 211) and Serfling (1980, page 18). Shao (1996) shows 
that in this framework any smooth function of estimates of 
totals is asymptotically normal. An estimate of a domain 
mean is one such smooth function. Likewise, any linear 
combination of different domain mean estimates is a smooth 
function of survey estimates of totals. For our purposes, the 
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bins form the domains and hence any vector of bin means is 
asymptotically multivariate normal. The asymptotic result 
used here depends on having a fixed number of bins. 
However, it can be incorporated in principle into a theory of 
the superpopulation parameters, as for example in the 
approach of Buskirk and Lohr (2005). 

Define m, (z) = (m,(z), my (z)' )’ as the survey esti- 
mator of m,(z) = (m,(z),m, (z)’)’. Using a Taylor 
linearization technique on (12) and letting ¢ denote a 
quantity approaching 0 and as 6 approached to @, we have 


—ii, (6)(B — B) = a(6) + U, (6) (i, (2) 
—m,(z))+||6- Olle, (13) 


where U(Q) is a linear sampling estimator of u(@) in (8) 
and is of the form 


(0) = Dy ia M,,) (x a Mx) 


kes 


7 > (x, —M,, ys (x, 


kes 


—M,, )Bw,; (14) 


u,(8) is the gradient of B obtained from (0); and 
U,(8) is a px(p+1)D matrix whose components are 
the first derivatives of (8) with respect to m,(z) . Denote 
by u,(0) and U,(8) the population parameters 
corresponding to 1, (0) and U, (8), respectively. 

In addition to the aforementioned regularity conditions, 
we impose the following conditions, letting MW denote a 
neighbourhood of the true value of the parameters of 
interest. 


Cl. lim, ,,,u(@)/N exists and is finite for all @ and 
N. 


C2. lim, ,,,Ug,(9)/N =H, and Hy, is of full rank 


and is invertible for all @ and N. 
C3. lim, ,,. U.(0)/N = H,(8) and H,(0) has a 


vo 


finite determinant for all @ and NV. 

C4. lim, ,,. 7 Var ,(i(8)/N) = V(a(8)) where Var, 
is the design-based variance and V(a(@)) is a 
positive-definite variance matrix for all @ and 


N. 
CS. lim 


vo 


N,/N =@, and lim, ,,.n/N = f with 
both w, and f are constants between 0 and 1. 

C6. Let A, =e’ (Z’K,,Z)'Z'Ky, be the popula- 
tion smoothing matrix; then lim, ,, A, exists 
ands finiteftor-a = 1b, (2): 

Cy ime = n Var (m,(Z)) = V(m, (Z)). 

C8. Matrices of population values Z’K,yZ and 


u,;(0) are invertible, as well as their sampling 
estimators Z’K,Z and G, (6). 


Survey Methodology, December 2009 


3.2 Asymptotic properties of B 


The proofs of all lemmas and theorems in this and the 
following section may be found in the Appendix. From the 
Taylor linearization results in (13), we know that the 
properties of B are dependent on those of 4(@), ti, (8), 
U ,(8) and m,(z); their properties are stated in the 
iene two Lemmas. 


Lemma |. If conditions C1 - 

as V — ©: 

1) Vn (a(6) — u(6))/N—> N(0, V(G0))); 

2) |ti,(0)—u,(8)|/N and |U,(8) — U,(8)| converge to 0 
in probability for ® and N; 

3) |a(8) — u(®)|/N converges to zero in probability. 

Lemma 2. Under conditions C5 to C7, Jn (a, (z) - 

m,(z)) = O, (1). 


Building on Lemmas 1 and 2, we have the asymptotic 
normality of B in Theorem 1. 


C4 are satisfied, we have 


Theorem |. Under conditions Cl to C7, assuming the 
parameter space contains a neighbourhood of the parameter 
of interest, we have as Vv goes to infinity: 


1) VnB- B)> N(@,V(@)) where V(B)=lim, ,..7Var ,B); 
2)|B — B| converges to zero in probability. 


To obtain approximate moments for B, we take 
expectations on both sides of equation (13), which yields 


E, (fi, (0)(B - B)) = E, (a0) 
+ E, {U,(8)[m,(z) — m,(z)]} 
+ E,(\|6 — @ll)e. (15) 


The assumption that the second moments of the estimates 
are bounded makes the last term of equation (15) vanish in 
the limit. Following along the lines of Binder (1983), we 
have 


E, (-a, @)) E, ((B-B)) =E, (a0) 
+ E,(U,(0)) E, {[in.(z)—m,(z)}}. 


The survey totals that define the vector i(@) and matrix 
ii, (8) are Horvitz-Thompson-type estimators and they are 
unbiased (Thompson 1997). Hence, E,(u(@)) = u(@) and 
E,, (fig (®)) = ug (8). Since u(8) is the estimating equation 
for the partial linear coefficients defined in (8), it is equal to 
a 1x p zero vector. Further, it has been shown in Bellhouse 
and Stafford (2001) that m,(z) is an asymptotically 
unbiased estimator of m,(z). Hence, —u, (@)E,, ((B - 
B)) = 0, or, based on the conditions that u, (8) is 
invertible and u, (0) is finite, we have E if (B) =B 
Taking the variance of both sides of equation (13) and 
using the approximated variance-covariance matrices of 
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u(8) and m,(z), we obtain the asymptotic variance of B 
as 


Var, (B) = u, (8) ' 
(Var, (a (@)) + U,(6)(A(J @Cov,, (x, y)) A”) U,(0)" 
+2(I, ®£)(£@C) A'U, (8))(u,(8))*, (16) 


where Var,(u(@)) is a px p matrix composed of 
variances of totals in the vector i(8) and Cov p(x, Y) 1s 
the variance-covariance matrix of the binned means of the 
parametric covariates and the response variable. The 
matrices J and &@ are the DxD unit matrix and the 
1 x D unit vector, respectively. Finally, we have 


I. ® Ay 0 0 0 
‘a 0 T,@A, 0 0 
0 0 0 
0 0 OME] STAT 
and 
Cov ,(t,, X,) Cova(tiaix.,), Cov, (t,.¥) 
Cz ; : . 


Cov,(t,, X,) Cov,(t,, X;) Cov ,(t,, y) 
where, sfotiey =z, p; t, is a Dx1 vector whose d™ 
entry 1S Likes, Wx Uj, (8) and A, = e’ (Z'K,Z)'Z'K 
fotde= dyunsd) 

Replacing 9, Var,(u(®)), Cov, (x, y), A and C by 
their sample estimators, we have the survey estimator of the 
variance of B: 


Var p(B) = =U, '(6) 
(Var (a (6)) + 0,(6)(A(J ® Cov, (x, y))A7) 0, 6)" 


+ 2(1, ® £)(£ @ C)A™U, (6)' ) (a5 6)", 
where A is the survey estimator of A and is composed of 
A, =e’ (Z'K,,Z)'Z'K 


3.3 Asymptotic properties of ¢(-) 


Define F = Y-—XB to be the sample estimator of 
R = Y- XB. A linearization around the population 
parameters, as well as design unbiasedness of domain 
means and B, leads to the asymptotic design aeons 
of r. Reexpressing ¢(z,), we have g(z,) = A,r. In 
A go we can expand CS Be using the Ae: series 
expansion that (I + G)' ~G+G’ --:- given that 
G is a symmetric and i matrix. Using the first two 
terms of the expansion, we can show that E (A nis 
approximately A,. Hence, we have the asymptotic design 
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unbiasedness of g(z,). With the same technique, the 
approximate asymptotic design-based variance of g(z,) is 
obtained as 


Var, (8(z,)) = AgVar,(F) Aj, 
where, given that Q = (1, —B,, ..., ell Hog 
Var, (F) = (Q ®1,)Cov, (x, y)(Q @I,)’ 
t X Var, (B) Xx’ 


— 2(Q @1,)Cov, (y, B)X" 
E ANwr 
- >) 2Q @I,) Cov, (x,, B)X’. 
j=l 
Given the estimated variance of ¥, namely 
Var p(F) = (Q ® 1p)Cov, (x, Y)(Q @ Ip)’ 
+ ¥ Var , (B) x’ 
— 2(Q @ I) Cov, (y, B)x” 
Ig A a A 
- ¥) 2(Q @1,)Covp(x, B)x’, 
j=l 
the estimated variance of g(z,) is Var p (2(z,;)) = 
A, Varp(F)A‘. 
The asymptotic normality of ¢(-) is also dependent on 


the normality of Yr, which is shown in the following 
Lemma. 


Lemma 3. Under conditions C1 to C7 and assuming that 
the dimension of ¥ is finite, we have as v goes to infinity 


JVn(¥-R) SNC, V(¥)), 
where 


V(r) =hm,,_,..7 Var, (1). 


Based on the asymptotic normality developed in Lemma 3 
and the estimator of the variance of ¢(z,), we establish the 
asymptotic properties of g(z,) in the following Theorem. 


Theorem 2. Under conditions C1 to C7, we have as v 
goes to infinity: 


I)|Z (22) — 8(2,)| > 9; 


=— d 
2)(& (Za) — 8 (Z,))/y Varp(&(z,)) > N(O, 1). 
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4. Simulation studies 


4.1 Design of experiment 


The simulation study implemented here was designed to 
illustrate the theoretical results in Theorems 1 and 2. We 
generated the data in a two-step process that mimicked a 
superpopulation approach to sampling. First, we generated 
the finite population and then the sample was selected from 
it. In particular, we considered a finite population of 
L = 500 clusters with M(= M,) =20,000 in each. The 
population observations for the measurement of interest y,, 
were obtained from the model 


Va By + Bix, + Box, 


Zi 


40 
+ 0.5exp +p, + &, (17) 


for i = 1, ..., 2 and j =1, ..., M where the error terms p; 
and ¢, are mutually independent with p, ~ N (0, 5,) and 
é, ~ N(0, o-). We set Go” = S, +o2 so that the 
intracluster correlation coefficient is p = o,/0°. Among 
the covariates in the model, both x,, and x,,, were treated 
as the parametric linear part of the model and z, as the 
nonparametric part. We generated the x,, from the 
Bernoulli(1/2) distribution and the x,, from the Uniform(0, 
1) distribution. The z, were generated from the age 
distribution of the Canadian population (according to the 
1996 census) for the 18 to 64 age range and were inde- 
pendent of the error terms. Results for the values B, = 1, 
B, = 2, B, = 3, o =3 and p =0, 0.2, 0.5 are reported in 
this study. A _ two-stage sampling design, with 
1(= 10, 25, 50, 100) clusters chosen at random from L 
and m(= 1,000) secondary sampling units chosen at random 
from each cluster of size M, was used for the study. For 
each sample size and value of p, the simulation was 
repeated 300 times. At the population level, we applied the 
bandwidth selection method from Fan and Gijbels (1995) 
and determined that the bandwidths for estimating the 
conditional expectations of X, and X, on z were 1.2 and 
1.5 respectively. When smoothing the residuals to estimate 
g(z), the bandwidth was 0.6. 


4.2 Results 


Using the generated finite population, we found that the 
census estimates were B, =2.01 and B, = 3.00. To check 
the design unbiasedness and efficiency of B, we calculated 
the simulated squared bias (Bias*), which is the square of 
the difference between the average of the simulated 
estimates and the census estimates. In addition, the ratio of 
the average variance estimates to the simulated variance of 
each estimator of a linear coefficients (RVar) is presented to 
show the validity of the variance estimator Var, (B). To 
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evaluate the normality of B, we standardized the estimates 
of linear coefficients using the empirical standard deviation 
and population value of B and graphed the quantile - 
quantile plots of the standardized values. 

Applying the semiparametric technique in Speckman 
(1988) to the model (17), we obtained census estimates 
g(z) for z = 18, ..., 64. To evaluate the design accuracy 
of ¢(z), we took the difference between ¢(z) and g(z) 
at each distinct point. The average of the squares of the 
differences over 47 distinct values of z is then reported as 
ABias*. Two mean square errors were computed to check 
the design efficiency of g(z) and convergence of 
Var p(g(z)). One of the mean square errors is the average 
of the estimates of the integrated mean square error 
(AIMSE), which is obtained by first summing the 
Var,(g(z)) over z = 18, ..., 64 for each simulation and 
then taking the average of the sums over the total number of 
simulations. The simulated integrated mean square error 
(IMSE) is another mean square error and was computed by 
summing up the simulated mean square error at each 
distinct point of z. The average of the ratios of the 
simulated mean of Var p(g(z)) to the simulated variance of 
&(z) (Reff) shows the convergence of Var p (¢(z)). In 
addition, we computed the coverage of the pointwise 95% 
confidence interval at each distinct point of Of. 

__The results on the properties of B, Var »(B), &(z) and 
Var p(g(z)) are found in Tables | and 2 and Figures 2 and 
3. Tables 1 and 2 show information about accuracy and 
precision of the simulated estimates of B and §(-). Figure 
2 gives the quantile-quantile plots of the sample 
standardized value of B,. Note that the quantile-quantile 
plots for B, behave in a similar way to those for iby Figure 
3 graphs the coverage of the 95% confidence intervals for 
g(-). In Figures 2 and 3, we only report the cases where 
1 = 10,25, 100 and p =0, 0.5. The overall performance of 
the estimators agrees with the theory in Theorems 1 and 2. 

Table 1 confirms the design unbiasedness of B. It also 
shows that as the sample size increases, the performances of 
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the estimates of the linear coefficients improve for all the 
error structures. In particular, the squared bias and variance 
of B decreases as the number of primary samples increases. 
The estimated variance of B gets closer to the simulated 
variance of B as the sample size increases; this confirms 
the consistency of the variance estimates of B. Comparing 
the variances and biases of B in the cases that p = 0.2 and 
p =0.5 to the case where p =0, we found that the 
intracluster correlation (cluster effects) did not affect the 
performance of B. This may be because the within cluster 
sample size was large. 

Observing Figure 2, we find that both the number of 
primaries sampled and the cluster effect play some role in 
the normality of B. In particular, when the primary sample 
size is low, for instance /=10, normality of the 
standardized B shows some deviation from the theory for 
both p =0 and p =0.5. When / increases to 25, we find 
that performance of B for p =0 starts to recover whereas, 
for p =0.5, there is no improvement until / = 100. 
Empirically, this finding suggests that when the number of 
clusters is low, we should not rely on the theoretical 
normality of the estimates of the coefficient; instead, we 
may want to use ¢ distribution to carry out the inference. 

As for the results of the nonparametric part of the esti- 
mation, Table 2 shows that the average estimated integrated 
mean square errors are very close to the simulated integrated 
mean square errors for all the sample sizes and error 
structures. Design unbiasedness is again confirmed with the 
average squared bias (ABias” ). The values of average ratio 
of the estimated variance to the simulated variance (RVar), 
which are close to 1 for all cases, are in line with the design 
consistency of the estimator of the variance of g(z). The 
integrated mean square errors of ¢(-) are influenced by the 
intracluster correlations. This can be shown by the fact that 
the approach to zero of both integrated mean square error 
and average estimated integrated mean square error is 
slower in the cases where p =0.2 and p =0.5 than in the 
case where p = 0. 


Table 1 ” 
Simulation results for point estimators of B 
p=0 
l Bias” Var Rvar Bias” 
(x10%) (x 10%) (x 10°) 

Bhat 57 1.07 1.13 3.12 

25 9:97 0.46 1.07 0.38 

50 0.54 0.21 1.08 0.13 

100 Oy? 0.13 0.96 0.019 

Bond 10 0.36 3.32 1.13 1.54 

25 0.64 os 1.10 2.40 

50 0.31 0.75 0.94 L27 

100 0.15 0.38 0.94 it 


p =0.2 p =0.5 
Var Rvar Bias” Var Rvar 
(« 10°) («10° = (« 10°) 

Ish 1.01 0.23 L.19 1:33 
0.44 1.08 0.30 0.53 0.98 
0.27 0.93 0.026 0.21 1.18 
0.11 1.06 0.039 0.13 0.98 
3.74 0.92 1.26 35 1.78 
1.34 1.06 0.14 1.42 1.03 
0.85 0.94 0.16 0.76 0.97 
0.38 0.98 0.072 0.33 1.03 
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Figure 2 Quantile — quantile plots for standardized B, 


2 =i 0 1 2 3 
Theoretical Quantiles 
(p =0.5, / = 10) 
Table 2 
Bias and efficiency of ¢(z) 
p l AIMSE IMSE_ ABias’  RVar 
(* 10-5) 
0 10 0.37 0.42 5.29 27) 
25 0.15 0.17 3.20 1.10 
50 0.074 0.086 3.29 1.09 
100 0.037 0.044 2.34 1.08 
0.2 10 2.95 B25 6.13 0.91 
25 122 ele Bali 1.04 
50 0.74 0.54 2.34 1.0 
100 0.26 0.27 7.08 0.98 
0.5 10 8.143 8.877 Beis 0.92 
US 3.155 3.073 6.56 1.03 
50 1.461 1.599 2.86 LES 
100 0.659 0.607 3.59 1.09 


The coverage of the point-wise 95% confidence intervals 
for g(-) in Figure 3 varies between between 85% and 96%. 
The coverage improves as the sample size increases. The 
performance of g(-) is, however, more sensitive to the 
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lower effective sample size caused by the intracluster 
correlation. In particular, the coverages of the 95% 
confidence intervals in the cases of p =0.2 and p =0.5 are 
smaller than the 95% nominal confidence level when 
/ = 10. The coverage improves as the number of primary 
sampling units increases for the cases of p = 0 and p =0.2. 
For p =0.5, the undercoverage is still present when the 
sample size increases to 100. It is also seen that at z = 18 or 
64, the coverages are higher even than the nominal level; 
this is because the boundary effect of the local polynomial 
regression estimation causes larger bias at the two 
boundaries of the data. For p =0.5, the effective sample 
size 1s low so that the boundary effect becomes severe, 
creating the downward spikes at 18 and 63. 

It is worth pointing out that although the size of the 
primary sampling units is large (1,000), the sampling 
fraction is very small (0.05). Hence, this performance of the 
estimates would not change even though the size of the 
primary sampling units is small. 
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Figure 3 Coverage of the 95% confidence point-wise intervals for g(z) 


5. Empirical illustrations 


We now returm to the example introduced in Section 1. 
For the purpose of illustrating the partial linear model, we 
examine the effects of age, gender, smoking status and 
physical activity on the body mass index (BMI) and the 
desired body mass index (DBMI). Similar to the measure 
BMI, DBMI is a derived variable for the question asking 
about the desired weight of a person. Since people stop 
growing for the age group for which we are interested, we 
use the actual height when calculating DBMI. We use age 
as the nonparametric covariate and treat the other factors as 


discrete variables. Since there are only 47 distinct points in 
the age variable, we bin the data set according to age. The 
bin size is set to unity such that there are 47 bins, with 
midpoints being 18, 19, ...,64. Among all the categorical 
explanatory variables, gender has two levels, male = 1 and 
female = 0; smoking status includes levels such as former 
smoker = 0, never smoked = 1, occasional smoker = 2, daily 
smoker = 3; and physical activeness is divided into three 
levels: active =0, moderately active = 1 and inactive =2. 
The regression models are (a) BMI = g,(age) + XB, + &, 
and (b) DBMI = g,(age) + XB, +€,, where X is the 
design matrix including all the indicator variables. 
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Table 3 lists all the survey estimates of the linear 
coefficients in the models (a) and (b). On comparing BMI 
by gender, we found that male BMI is higher. Using former 
smoker as the base category, the coefficients of smoking 
status are all negative and significant, which suggests that 
former smokers tend to be heavier than people with other 
types of smoking status. The estimates also indicate that 
inactive people have higher BMI. With respect to the 
DBMI, p -values suggest that most of the life style related 
factors are not significant. 


Table 3 
Results for semiparametric regression models (a) and 
(b) (Values in the parenthesis are the standard errors) 


A 


Variable B, p-value B 2 p-value 

Gender 1.45 0.00 2.80 0.00 
(0.05) (0.05) 

Never Smoked -0.45 0.00 -0.06 0.34 
(0.10) (0.06) 

Occasional Smoker -0.31 0.04 -0.00 0.96 
(0.17) (0.10) 

Daily Smoker -0.61 0.00 -0.12 0.03 
(0.09) (0.06) 

Moderately Active -0.33 0.00 -0.07 0.24 
(0.09) (0.06) 

Active -0.50 0.00 -0.14 0.07 
(0.09) (0.09) 
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In Figure 4, the estimated functions of age, ¢,(Age) and 
&,(Age), and their confidence bands are plotted versus 
different ages. It is found that, in both cases, the BMI and 
the DBMI are increasing functions of age. 

Figure 5 gives the estimated functions of age, ¢, (Age) 
and ¢,(Age), for active and moderately active people. If 
we look at the age effect for female and male separately, we 
find that for females who are either active or moderately 
active on average the DBMI is lower than the BMI, whereas 
males with the same intensity of physical activity desire to 
be heavier before age 21. In addition, we also compare the 
age trends in the BMI and the DBMI for both the females 
and males. Due to the inconsistency between the female and 
male trends, we can conclude that there are interactions 
between the gender factor and age. 


6. Conclusion 


With the assistance of a partial linear model, we extend 
semi-parametric regression techniques to complex survey 
data. Asymptotic properties of the survey estimators are 
developed. Computation of the variance estimates of both 
the linear coefficients and the regression function rely on the 
variance estimates of survey totals and means. Provided that 
we obtain the required variance estimates of survey totals 
and means, we can apply this method using standard 
statistical packages. 
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Figure 4 Estimated age trends in BMI and DBMI with 95% pointwise confidence intervals 
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Age Trend in BMI and DBMI for Male 


20 30 40 50 60 


Figure 5 A comparison of estimated age trends in BMI to DBMI for both female and male who are active and 


moderately active 


In the partial linear working model, we assume that there 
is no interaction between the parametric component and the 
nonparametric component. However, the empirical example 
of the age trends of the body mass index has illustrated that 
this assumption needs to be checked. In future work, we will 
relax the no interaction assumption. A direct approach to 
modelling interaction terms is to let the nonparametric 
component appear linearly in the interaction term. That is, 
we define the partial linear model as 


y = G(z) + XB + XA (z) +e. 


By testing the departure of H(z) from zero, we can detect 
the existence of interaction. 

When estimating conditional expectation on _ the 
nonparametric components for indicator discrete random 
variables, we propose to use generalized linear or additive 
models to conduct the estimation. 


Appendix 


A.1_ Proof of lemma 1 


Observing that entries of (8), 0,(0) and U, (0) are 
either sample totals or ratios of sample totals, we can apply 
Lemmas 1.2.5 and 1.2.6 in Wang (2004) to establish this 
Lemma. 


A.2. Proof of lemma 2 


Each entry of m,(z) is just an estimated regression 
function with the local polynomial technique developed by 
Bellhouse and Stafford (2001). Theorem 2.2.1 in Wang 
(2004) shows that m,(z) is root-n consistent. Hence, since 
the dimension of m,(z) is finite, we can show that 
Vn(ti,(z) — m,(z)) is bounded in probability. 


A.3 Proof of theorem 1 


Since for the true 8, we have u(@) = 0, we can rewrite 
equation (13) as follows: 


_vnii, (8) OE 
N 


a — u(6)) + 0,0) (rig a ‘ tO) 


it: Vnig — Olle. 
N 


The standard argument in Rao (1973, page 387) yields 


‘ P 
Jn/N ||6 - O\le > 0. 


Using the condition that the sampling fraction f = n/N is 
constant as v goes to infinity, we have, 


Vn(B - B) = Ce) 
N 


bac 


FP) - m0) | 


Following from the results in Lemma 1, both (i, (8)/N Ve 
and Lupe _ (9) converge to their population values in proba- 
bility. erase 2 indicates that the vector Jn n (mi, (z) - 
m,(z)) = O,(1). Thus, (fVn/n)(m,(z) - m,(z)) con- 
verges to a zero vector in probability as n goes "a infinity. 
Finally, from the normality of Jn(a(®) — u(@))/N stated 
in Lemma 1, we use the Slutsky Theorem to show the 
asymptotic normality of B. 
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A.4 Proof of lemma 3 


Given that F =y—-xXB, we have Vn(F -R) = 
Vn [(y - XB) — R]. Based on Theorem 1, we know that in 
the limit as v goes to infinity, B converges to B in proba- 
bility. Hence, we have Jn(F — R) = Vn((y — XB) — R). 
The d" entry of (y — XB) is, 


Ng Ope re XndB, a 


] 
a ake W, Vee Oy ee Bo: 
d 


That is, (Y — XB) is merely a vector of estimated binned 
means. Using the result from Shao (1996) on functions of 
sample means and “Cramer-Wold device” results found in 
Serfling (1980, page 18), we see that J/n(y — XB — R) 
converges to a random vector distributed normally. Thus, 
using this indirect Slutsky idea, we have proved the 
normality of J/n(F — R) = Vn[(¥ — xB) - R]. 


A.5 Proof of theorem 2 


The proof follows the same argument that g(z,) is a 
function of domain mean and proportions as does in the 
proof of theorem 2.2.1 in Wang (2004). 
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